It’s been a while since my last blog, and frankly that is a good thing for two reasons. Firstly, I must admit that work has been keeping me rather busy and that is always preferable to sitting idly twiddling your fingers. But second and most importantly, it is through this last project that I was working on that I had my moment of inspiration for this blog.
So, I am working on this Genesys project ….as we do….performing the integrations with Genesys to use POP3 over SSL, LDAP over SSL, Kerberos Single Sign-on, and UCS Database data synchronisation….which I do….when I am suddenly asked to help with validating firewall changes on the customer side. OK…not a problem, I mean we’ve all been there with these enterprise solutions. They come with a suitcase full of port connectivity requirements and getting them nailed is a crucial part of any integration work.
So I turn up and am ready to go through the connectivity testing with the person who is coordinating this change with the customer, and believe me this end customer is a large multi-national so I am expecting some amazing toolsets that they probably use to do their validation of firewall changes. Anyway, I ask him, “Hey, dude….what tools will we use to validate the firewall?” (Yes I say ‘dude’ from time to time). To which his reply was “telnet”.
Let me put this in perspective, this is a multi-server, multi datacentre, high available, fully redundant Genesys deployment for which the number of unique firewall requests came to around 200+. Nearly a quarter of these firewall port change requests were actually ranges of a 1000 ports, and the plan was to work through each one by typing in the command prompt ‘telnet hostname port’….200+ times.
What I said next wasn’t something I am proud of but nevertheless the ‘dude’ whom I was working with wasn’t that bothered about it. His words were “This is not penetration testing we are doing, just validating port connectivity. We have always done it this way as it has no reliance on third party tools”
Look I know firewall port validations can be a pain and going through them is something we just have to do in our line of work, but why do some people choose to switch off their brains and never ask themselves, “Is this really the best way to go about doing what needs to be done?”
Let me have a Go!!
So that is exactly what I will try and do here. Now I am going to be honest, I am not a network engineer, or a penetration tester, or a security expert by any stretch of the imagination. But what I do have is 17 years of experience in this industry where I have worked as a software engineer, a solution engineer, Senior Consultant, Enterprise Architect and even a Project Manager. Therefore with all my projects I try and take the lessons I have learned in all my various incarnations and come up with an approach this is hopefully not only efficient, but also in line with the level of governance and traceability that most major enterprises expect.
When it comes to firewall port validations there are really two phases which you need to worry about.
1. Planning and Coordination
Planning and Coordination
This is not just for Project Managers. The successful delivery of any project depends on the team striving to help the Project Manager to get authorisation, coordinate, and fully document any work you carry out. Here are my tips to ensure this phase is done smoothly
1. Submit All Firewall requests in a single go: Most big enterprises have a long turnaround period for firewall change requests. The process of submitting, evaluating, scheduling and executing the changes in an organisation can be anywhere from 1 week to 6 weeks. Doing firewall port request changes in multiple batches can seriously affect project timelines as a result. Find out about the customers processes, their request templates, the subnets you will be affecting early on in the process so you can save time in the end.
2. Corporate Policy on Port Connectivity Testing: Does the end customer have any requirements that need to be adhered to by third parties? Do they have in-house teams and tools you can utilise? If not then can you use your own laptop and tools and connect to their network? Will they provide a corporate machine for you to do the testing? Are you allowed to install your own tools on this machine? All the above are crucial to help you with your strategy for the execution phase.
3. Get Approval for the testing: This is crucial. You need to get authorisation from the end customer through their internal approval processes to carry out the actual testing. The last thing you want is your testing to be flagged as a possible intrusion or DDOS attack which will make the hair on the back of the necks of any network team stand up. Clearly outline the subnets and ports you will be targeting, the source and destination endpoints, the speed at which you will be carrying out the tests and what tools you will be using. Failure to get this approval can cause serious headaches down the line.
4. Arrange firewall team cover for the testing period: Make sure that a resource is allocated from the end customer’s network and firewall admins team for the duration of your port connectivity testing. It is always quicker and more efficient to resolve missed ports or erroneous configuration straight away after a change request has been implemented as opposed to later, which usually end up requiring another change request to be put through.
During this phase you are actually going through the validation of your firewall port change request. The second point of my Planning and Coordination phase will feed into the actual approach you should be taking to get this done as smoothly as possible. Personally, I am a big fan of Nmap and PowerShell, and is it turns out they both support scripting and that my friends translates to you pressing enter and walking away as opposed to typing telnet commands one after the other.
The Nmap Scripting Engine (NSE) is one of Nmap’s most powerful and flexible feature, which allows users to write simple scripts using the Lua programming language to automate a wide variety of networking tasks. To start your journey on creating your own scripts go to https://nmap.org/book/man-nse.html
When it comes to PowerShell scripts my favourite is the function Get-PortState, the script for which you can get here. I really liked this script cause I could simply copy it into a PowerShell command console as a standard user and then call the function inside a script to test all my ports. Its relatively simple and you can learn more at http://www.powershelladmin.com/wiki/Check_for_open_TCP_ports_using_PowerShell
The choice of which tool and approach to use will largely depends on the environment you are working in. My advice, always go with Nmap if you can but if circumstances force you, then PowerShell is a worthy alternative. Use the flowchart below to help you decide.
But what did I do?
In my case working with the ‘dude’, after about an hour and half, we only got through 25% of the tests. That was my queue to press pause on the torture and go in search of some sanity. As with this customer we weren’t allowed to use our own machines, I managed to find the PowerShell scripts for Get-PortState and prepared the scripts for the remainder of the tests. I did modify the actual PowerShell slightly to introduce a 5 second delay between each test, but other than that, I just loaded the script on the laptop, pressed ‘Enter’, and went to get some coffee….Nice!!!
I rolled off the project a while back, but I found out recently that my scripts with the delays was used by this particular partner on another project much to the relief of the engineering team and end customer.