How to do a HTTP POST with Curl

This page proves there are people out there who forget about the same things I forget.

Here's the POST I wanted to run today.

curl -d "platform=CHANNEL&processor_target=CERT&charge_type=PING" http://app01.qa.paygateway.com:31080/Quicksilver/

Cisco ASA/PIX Bandwidth limiting

Today, I got a chance to setup some bandwidth limit on our Firewall. The goal was to choke the speed of traffic going to our backup server to 250Mb/s. As our backup traffic goes through a firewall here's how I did it on the my ASA5520s

First Define the traffic I want to choke using an access-list:

access-list backup_traffic extended permit ip any host BACKUP01

Then create a policy map to and set the speed

policy-map backup_traffic
class backup_traffic
police input 250000000

Then apply the policy map on the interface

service-policy backup_traffic interface outside

All done! That was easy. There's a lot more you can do in terms of QoS on the PIX/ASA. All this information can be found on the cisco site.

Linux Search and Replace multiple files

I don't know how many times I've forgotten how to do this but here's the summary.

Objective: I have some text I want to modify in many files.

Solution: Using SED in a bash FOR loop

for a in `find . -name '*filename*'`; do sed 's/text1/textx2/g' $a > $a.bk; mv -f $a.bk $a; done;
.
In your FOR loop
Step 1: Find the files you want to modify
Step 2: Use sed to search and replace the contents and redirect it into a new file
Step 3: move the new file back to the old file
Step 4: close your loop with done

The trick I forgot here is I can put as many commands as I want in a FOR loop by using the ';' delimiter.

Also, as the '>' - redirect cannot be used to overwrite the current open file the trick is to split the operation into two; write to new file and then move back to old file.

Cisco ASA/PIX icmp handling

Just a quick note:

Internet Control Message Protocol (ICMP) pings and traceroute on the PIX Firewall are handled differently based on the version of PIX and ASA code.

Inbound ICMP through the PIX/ASA is denied by default. Outbound ICMP is permitted, but the incoming reply is denied by default.

Pings Inbound
Pings initiated from the outside, or another low security interface of the PIX, are denied be default. The pings can be allowed by the use of static and access lists or access lists alone

Pings Outbound
There are two options in PIX 7.x that allow inside users to ping hosts on the outside. The first option is to setup a specific rule for each type of echo message.

For example:

access-list 101 permit icmp any any echo-reply
access-list 101 permit icmp any any source-quench
access-list 101 permit icmp any any unreachable
access-list 101 permit icmp any any time-exceeded
access-group 101 in interface outside

This allows only these return messages through the firewall when an inside user pings to an outside host. The other types of ICMP status messages might be hostile and the firewall blocks all other ICMP messages.

Another option is to configure ICMP inspection. This allows a trusted IP address to traverse the firewall and allows replies back to the trusted address only. This way, hosts on all inside interfaces can ping hosts on the outside and the firewall allows the replies to return. This also gives you the advantage of monitoring the ICMP traffic that traverses the firewall. In this example, icmp inspection is added to the default global inspection policy.

policy-map global_policy
class inspection_default
inspect icmp

For more detailed info visit: here

Linux 101 - TOP command

Top is amongst the most comprehensive and informative process display command in Linux. There's no reason for a seasoned Linux sysadmin like myself not to know it! Top can display all the information you need to know about processes and systems information in a sorted manner.

In this post I want myself to be reminded of only the MOST USEFUL commands used in TOP and the rest I'm going to forget them!

h - Help - Yes, use it if you're as forgetful as me.
M - Sort by memory usage
P - Sort by CPU usage
c - Display path and exact command
z - Change colours - I hate Black & White
u - Display only processes owned by a specific user, enter nothing for All
q - Quit or you can crtl-c to kill it.

I think that's really the minimum options a sysadmin should know. I would refer to the MAN page if I was ever stressed to find out more.

Watch the latest videos on YouTube.com

Cisco Spanning Tree protection mechanisms

Today, I wanted to remind myself of all the funky little Cisco spanning protection features. I found this neat picture that summarizes it all:

Troubleshooting PIX/ASA site to site IPsec VPN

I'm a person, who constantly makes typos, so troubleshooting has become a very part of me. Especially with such complex VPN configurations, there's bound to be some mistakes somewhere.

Like any other technical problem, the first thing is to understand the underlying technology. A site to site IPsec VPN consists of two phases; Phase 1 - IKE exchange and Phase2 - Establishing the ipsec tunnels. It is important to figure out which part of the negotiation the VPN is failing at.

However, before all that, we should check if the traffic is getting to the firewall in the first place. To do this we should turn on icmp debug on the firewall.

debug icmp trace

If you are logged in to the firewall via ssh or telnet, unlike the console you may find that you want to redirect the output of the debug to the screen. To do this enter:

terminal monitor

Once that is done, you can now start pinging the system you want to reach at the other end. Observe the icmp packets as it reaches the firewall, it should first reach the firewall and then natted to the global IP that was assigned in the VPN. If this is not true, check your nat and global acls and statements. You may want to use these statements to investigate your config

sh run nat
sh run global
sh run access-list acl-name

If the traffic is being properly natted you are safe to turn off icmp tracing as you know the traffic is reaching the firewall and being translated properly.

The next step is to verify that PHASE 1 negotiation is complete. Make sure to keep pinging the target host as it would ensure that the VPN tunnel is staying active.

sh crypto isakmp sa detail.

This will display if the sa - security association has been established. From the output you will see the status of the negotiation, if it succeeds will you see that status has become active and your sa time to live should start to decrement.

If not, turn on debugging for the phase 1:

debug crypto isakmp

Verify that there are no mismatches in the configuration as usually that should be the problem. Do a:

sh run crypto

Verify that the config does match the configuration provided to you on the other end. You may also find that the admin on the other end may have their configuration misconfigured. Try to ensure that your configuration is correct before blaming others as you don't want to to look stupid.

If the phase 1 configuration is complete, then you can move on to troubleshooting phase 2. Much like phase 1 you want to sh the crypto status.

sh crypto ipsec sa detail id-number

The id number here is the crypto-map sequence id number entered for the specific tunnel. Verify that something is displayed. If nothing is displayed then there is likely a problem with the configuration of the phase 2. Check that the ipsec transforms sets are matching as this was the problem that I ran into.

Once that is all done and both phase 1 & 2 are complete then the tunnel should be up. Now, if you have read carefully, all this requires that you ensure that the tunnel is up by launching a consistent ping against a target host at the other end. However, if you realize that both phase 1 & 2 have been established but you do not see any icmp replies, this may be to do with the natted address that was provided to the other side. What usually happens is that the other end would have an access-list that would only allow traffic from one specific natted ip. If this is wrong, you will not get any response even if the tunnel is up. Also, make sure that the outgoing acls on the firewall is to allow icmp going out. Depending on the corporate policy one may have defined only specific IP going out.

One last tool to troubleshoot, use the ASDM! This neat GUI will tell you all the information at a glance. It is a useful too for viewing your configuration. Also, try using the packet tracer on the ASDM. It will display in detail how the packet traverses each component in the firewall and show you the likely place to look at in your settings.

That's it for troubleshooting VPNs. These are the problems that I ran into when I set it up. I hope when I look back at this post I can remember what pains I have gone through and ensure that I follow these steps to avoid them!

Cisco PIX/ASA site to site IPsec Tunnel

I've created less than a hand full of VPN tunnels so far in my career. How often do you get a chance to create new VPN tunnels? Well today, I got a chance to create 3 customer site to site VPNs at once!

Speaking of VPN tunnels, how many know the commands for creating an IPSEC site to site VPN without referring to documents? In my job, everything has to go through a documented procedure in creating change requests, so CLI has become the only method we use in making Firewall or Router changes as it is the most straight forward to document. However, I must say using the ASDM in creating VPNs is much easier and much less error prone.

Today, I'm going to write down what my brain went through creating these VPNs and what sort of things I want to remember when I come to read this again.

First of all, we need a bunch of info and prerequisites when creating a VPN. It is critical that we communicate clearly so both ends have the correct information especially when working with a network admin on the other end of the pipe.

So what do we need to decide first?

- We should NAT to a public IP that we own for them to filter our traffic. There are two advantages:
1) We are not tying up our Outside interface IP incase we move it.
2) We are more flexible in movin that natted IP onto another FW if we want to in the future!

- Obtain the Outside IPs on both ends to establish the tunnel. This would be the outside interface IPs on the connecting devices.

- The Source and destination domains for the VPN. This would be the source IP we are encrypting our traffic to (As explained we should NAT this to a range or one IP depending on customers, unless we are making an internal site to site VPN in which case we may not want to NAT) and destination IPs - the hosts we want to connect to on the other end

- Phase 1 IKE exchange information
authentication method (usually pre-share)
encryption algorithm (des,3des,aes, etc)
hashing algorithm (md5,sha)
DF group (2,5...)
SA lifetime (86400)

- Phase 2 IPsec tunnel information
Peer IP
encryption algorithm
hashing algorthm
DH group

Watch the latest videos on YouTube.com

Once we have all this information we can start creating our VPNs.

There are two types of Cisco devices we can terminate our VPNs to; Router or Firewall. Both have advantages and disadvantages. I prefer creating site to site VPNs on routers because on routers VPN tunnels are created as VTI - Virtual Tunnel interfaces. These interfaces are just like router interfaces and are much flexible in nature. ASA/PIX are good with creating Remote Access VPNs as you can slap on and define the access policies much better. However in my scenario, I only had an ASA and creating a VPN tunnel on that is what I want to record down today.

Step by Step approach:
1) Backup your config!
2) Define and configure your PHASE 1 ISAKMP policy

crypto isakmp policy 30
authentication pre-share
encryption 3des
hash md5
group 2
lifetime 86400

*note: the policy number can be any number, it's just a priority number . The IKE exchange will go through all the policies you have and hit on the first one it matches on the other end.

3) Define the PHASE 1 ISAKMP attributes by creating a tunnel group


tunnel-group x.x.x.x ipsec-attributes
pre-shared-key *

Note: The x.x.x.x here MUST be the peer IP address.

4) Define the PHASE 2 transform set

crypto ipsec transform-set TUNNEL_ESP_3DES_MD5 esp-3des esp-md5-hmac

In this case, I gave the tunnel name: TUNNEL_ESP_3DES_MD5 which closely describes what it uses: ESP + 3DES + MD5 + HMAC

5) Create the policy NAT statement and create the NAT

access-list acl permit ip source mask destination mask
global (outside) 3 ip
nat (inside) 3 access-list acl-name

Here we defined the traffic we want to be NAT using a combination of access-list for and the IP to nat to. Note that the 3 here is the 'id' I'm using to match the corresponding NAT statement. FYI, you can't assign more than one acl to a nat statement (I tried it)

6) Define the Encrypted traffic using an access-list

access-list acl extended permit ip host natted dest mask

Note: We are encrypting the traffic AFTER it's been natted so the access-list would fail if it is refering the the source internal IPs.

7) Lastly Define the crypto map

crypto isakmp enable Outside
crypto map map-name id match address
crypto map map-name id set pfs group2 <---if DH G is required
crypto map map-name id set peer 206.253.178.46
crypto map map-name id set transform-set TUNNEL_ESP_3DES_MD5

Make sure that you enable the isakmp for the interface used for establishing the connection. There can only be one crypto map per interface and hence the 'id' comes into place. When you create more than one vpn on an interface you will need to specify a different 'id' for each tunnel.

I guess that's all there is to configuring the tunnel. I will be posting some ways to test and debug the connection in my next post.

The Linux "find" command

In my Linux life, I find myself trying to search for files quite frequently. All distributions comes with a very handy command called 'locate' which helps you search for the file. What it does is Linux keeps an index and database of all the files and locations and this command would search inside that database to try to look for the file you want.

However, what you might find out is you will need to keep that database up to date before you can search anything new. In order to keep that database updated you have to run a command called 'updatedb' and it will churn away indexing your files. The down side with this is that it often takes a long time for it to come back (depending on how big your file system is). This is when the good old 'find' command comes to use.

The find command is pretty simple to use. Here's the syntax:

find path -name 'filename' path arguments

Here are some examples

find . -name 'foo'

What this command does is it searches your current directory for the filename that contains the word foo.

Other varations include

find / -maxdepth 2 -name 'foo'

This command searchs for foo from / directly with a maximum of 2 directories deep.

After all that what I wanted to remind myself was this command

find . -maxdepth 1 | sed 's/\.\///' | xargs -i chown -R {}.{} {}/

AHA! mumbo jumbo. What this command does is it searches the local directory, prints out the filenames and then strips out the ./ and then changes the ownership of the files(directories I should say) in question to the name itself?

So you might think, why would I do that? Well today I came across a system that had some screwed up home directories where ownership of directories did not belong to who they are suppose to. This command fixes all that, find find the directory name and changing the ownership back to it's real owner (which happens to be the name of the directory)

This command combines a series of linux command parsing, but what I want to make note is the 'xargs' command. This neat little command feeds from stdin and allows you to form a command which takes that input as an argument. In this very special case, it's especially useful in combo with the find command as it can feed input from the results of the search into a powerful chain command. One thing to make note is that the find command itself comes with '-exec' option which does the same thing, however in my case I had piped it once so I couldn't use the -exec command to do it.

Well to sum up, I've used this command many times and I think it deserves to stay in my memory for future use!