Musings on software development…

IPSEC from a Cisco 800 Router via Netgate pfSense into an Amazon VPC

Hi,

pfSense is awesome and I was excited to see that there is a dedicated pfSense product from Netgate available on Amazon AWS. Cutting a long story short, I finally figured out how to route one private network A (my home network – 192.168.75.0/24) which had a Cisco 800 series router on it, into another private network B (10.0.0.0/16) – with the important note that network B is inside a protected Amazon VPC space (a test vpc that I use for playing around with).

Background

I created an Amazon VPC using the second option, i.e., VPC with Public and Private Subnets. No hardware configuration was selected as I want to create the IPSEC tunnel using software only (well, without a reliance on the amazon hardware). The reason behind this is that my IP address (on my home network side) is dynamic and creating an Amazon VPC using the third option, VPC with Public and Private Subnets and Hardware VPN Access, “locks” your VPC VPN endpoints into the IP address of your network at the point of creation. Yes, it’s possible to then re-edit your Cisco config, but I didn’t want to keep doing that – instead, by choosing the software only route (option 2) and by using pfsense – which supports the endpoint having a DNS-resolvable name, it’s possible for me to bring up my tunnel anytime I want simply by trying to connect into the Amazon VPC – the tunnel will resolve the endpoint hostname during creation (phase 1) time.

Amazon Configuration

After selecting the VPC type that I wanted, Amazon created two VPC subnets, a 10.0.0.0/16 subnet which is public and therefore potentially visible to the outside world and a 10.0.1.0/16 subnet which is internal which no-one can see from the outside world. With these subnets in place, it’s time to get to work.

Firstly, install Netgate as recommended. Ensure that you create your instance your public VPC subnet (the 10.0.0.0/16 subnet). You need to have the ability to assign an Elastic IP to it and for it to be publicably reachable and therefore the instance creation has to be in the public subnet.

Then setup your your IPSEC phase 1 and phase 2 configuration:

IPSEC Phase 1 and Phase 2 Overview

Phase 1:

IPSEC Phase 1 Configuration

Phase 2:

IPSEC Phase 2 Configuration

So far, so good.

Next, go into Firewall and select Aliases and insert both your home network and the Amazon VPC network into the alias list:

Network Aliases

NAT Configuration

Then under Firewall, NAT – be sure to choose Manual Outbound NAT rule generation (AON – Advanced Outbound NAT) option:

NAT Overview

Notice how your aliases “Networks_to_NAT” is in the Source column. This is good.

Under Interfaces, WAN – ensure you have Block private networks unticked:

WAN Interface Configuration

Next, you want to assign an Elastic IP address to your Netgate instance:

Elastic IP Configuration

Also ensure you have the security group setup correctly for your instance:

Security Group Configuration

That’s all from the Amazon VPC side of things.

Cisco Configuration

Now the Cisco side of the story. Instead of walking step-by-step through this, I’ve got an edited runtime configuration that you can examine – naturally I’ve removed the secret key and the actual endpoint I use, but you can fill your information in where indicated:

!
! Last configuration change at 20:12:17 UTC Thu Feb 20 2014 by root
! NVRAM config last updated at 08:42:04 UTC Sun Mar 2 2014 by root
!
!
crypto ikev2 nat keepalive 5
!
crypto isakmp policy 1
encr aes
authentication pre-share
group 2
lifetime 28800
!
crypto isakmp key IPSECPASSWORD address XXX.XXX.XXX.XXX no-xauth
crypto isakmp keepalive 10
!
crypto isakmp nat keepalive 5
!
crypto ipsec transform-set strong esp-aes esp-sha-hmac
mode tunnel
!
crypto ipsec df-bit clear
!
crypto map amazon-vpn 30 ipsec-isakmp
description Amazon VPN
set peer XXX.XXX.XXX.XXX set transform-set strong
set pfs group2
match address acl-amazon-vpn
!
interface FastEthernet4
ip address 192.168.75.249 255.255.255.0
ip access-group acl-inside out
duplex auto
speed auto
crypto map amazon-vpn
!
interface Vlan1
description $ETH-SW-LAUNCH$$INTF-INFO-HWIC 4ESW$
no ip address
ip tcp adjust-mss 1452
!
ip route 0.0.0.0 0.0.0.0 192.168.75.1
!
ip access-list extended acl-inside
permit ip any any
!
ip access-list extended acl-amazon-vpn
permit ip 192.168.75.0 0.0.0.255 10.0.0.0 0.0.255.255
!
end

Right, once you’ve merged in a configuration similar to the above to your Cisco running configuration, you should be able to bring up the tunnel by pinging the remote private IP address endpoint (which in my screenshots is 10.0.0.200), i.e.,

cisco881#ping 10.0.0.200
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.200, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 16/18/20 ms
cisco881#

And here is me pinging into the private subnet (remember that 10.0.1.0/24 is private and not accessible from the outside world – however, now through the magic of IPSEC and pfSense, it is not accessible via the tunnel!)

cisco881#ping 10.0.1.10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.1.10, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/20/20 ms
cisco881#

I hope this short, brief and totally missing somethings post is useful. Shout up if I’ve forgotten anything!

-=david=-

Written by dharrigan

March 2, 2014 at 9:55 am

Posted in life

Vim and Buffer Bliss

Today I finally solved a problem that has been bugging me for ages. I would select some text (in a term or in an application) and want it to be pasted into the King of All Programs, Vim. I tried all the suggestions about setting the clipboard to be unnamedplus and so on with no joy. I even installed parcellite (which will remain installed, great bit of software) but I didn’t have much success. Finally, the answer came from a 1 minute discussion on #vim on #freenode.

Normally when I install a new (xubuntu) system, I simply do “apt-get install vim” and roll with that.

The solution is to “apt-get install gtk-vim” since it is that version that installs +clipboard and +xterm_clipboard support! (You can verify this by executing “vim –version” and examining the output). I was hesitant at first to install gtk-vim since I thought that this would force me to use gvim – and as I’m a total dyed-in-the-wool CLI guy, the thought of using a GUI vim wasn’t very appealing to me. Thankfully this is not the case as installing gtk-vim doesn’t force gvim to be used! w00t!

After installing gtk-vim, I can now select some text from another term or an application (using the mouse or if the application supports it, CTRL+v/x) and paste it into vim using “*p (or “+p). Fantastic! This also works vice-versa in that if I yank some text in vim (using “*y or “+y), I can now paste it into another program or term – all thanks to gtk-vim and parcellite working beautifully in tandem together!

Therefore, the steps to ensure all this works are:

1. Install gtk-vim
2. Install parcellite
3. In parcellite, ensure “Use Primary (Selection)” and “Synchronize clipboards” are both ticked.
4. In your .vimrc, ensure you have “set clipboard=unnamedplus”

That’s it. If you do this, then you’ll have copy and pasting ecstasy!

-=david=-

Written by dharrigan

October 11, 2013 at 8:53 am

Posted in software, vim

dnsmasq goodness

At present, I’m working from home. In order to access certain services, I normally VPN into the office network and leave the VPN running all day. Unfortunately, doing so cuts off DNS resolution from my machine to other computers running on my home network (I run my own bind9 server at home). This is because, on connection, the office VPN (which I configured :-)) pushes to each client, routes, ntp servers and also the office DNS server – all well and good – but how on earth do I now access my own home machines (without resorting to lame-ass /etc/hosts entries?) since the office DNS knows nothing about my home network machine names!

What to do? The answer is the marvelous dnsmasq program and a very simple configuration! I assume, of course, you have (or will about to have) dnsmasq installed.

First step is a simple change to my local resolv.conf, to read:

nameserver 127.0.0.1
search home.network office.network

This causes all DNS resolution to be forwarded to dnsmasq (which we will shortly configure to listen on 127.0.0.1). Additionally, I configure two search domains so that simple hostnames like “bob” or “bamboo” or “fish” will be changed to “bob.home.network” or “fish.office.network” depending on the results of queries to DNS.

Second step is to setup dnsmasq and because I’m using the wonderful, amazing, beautiful and oh-so-loveable Debian, I create a dnsmasq configuration file in /etc/dnsmasq.d and name it 00default (the name doesn’t matter at all – could be called wibble for all it cares).

The contents of this dnsmasq configuration file are:

server=192.168.75.3
server=/home.network/192.168.75.3
server=/office.network/10.10.10.3
listen-address=127.0.0.1
bind-interfaces

Whenever the VPN is up and running:

Firstly, all queries for all addresses are sent to my home DNS, i.e., http://www.google.com is looked up from the DNS listening on 192.168.73.3. This is the default behaviour and allows me to access external resources as normal.

Secondly, if I query for a machine called bob (which exists on my home network), my computer first tacks on home.network (from my resolv.conf), and queries the nameserver at 127.0.0.1 – which is where dnsmasq is listening. Dnsmasq in turn hands off to 192.168.75.3 since the query now matches the domain name of bob.home.network (notice the home.network in the configuration file). My locally running DNS responds saying “Yup, I know that address for that computer and returns with 192.168.75.2.” All honkydorey and sparkles.

Finally, if I query for a machine called bamboo (which does not exist on my home network), my computer first tacks on home.network, queries my local DNS (same rules as before), but that fails to return an address. An error is returned, resolv now tacks on office.network and dnsmasq works it’s magic by now querying the DNS server at 10.10.10.3, since the query now matches the domain, i.e., bamboo.office.network. That DNS server in the office knows about bamboo.office.network and responds with the correct IP address. Ah, all rainbows and unicorns.

Lovely! Thank you dnsmasq!

You can leave dnsmasq running all the time. It won’t resolve any office network names if the VPN is not up and running (since it cannot connect to the office DNS at 10.10.10.3). Oh, and my domain names (home.network and office.network) are only examples folks. They are not the real domain names. Change those to suit your particular setup.

I hope this is of some service to people!

-=david=-

Written by dharrigan

November 3, 2012 at 10:39 pm

Posted in debian, linux

Tagged with , , ,

Blag Blag :-)

w00t!

w00t! :-)

Written by dharrigan

July 15, 2012 at 5:29 pm

Posted in life

Delegating local subdomain BIND9 queries to Amazon Route53

Hello!

Recently we have started to use Amazon AWS heavily. It’s a truely great service. One of the tasks we need to accomplish is to join our locally running BIND9 DNS server to instances within our Amazon Virtual Private Cloud. We wanted to basically allow Amazon Route53 (their flexible DNS service) to resolve instances running with Amazon. So, how do we do this? In the end, it turned out to be really simple!

Let’s say you have an instance at 172.16.100.5

And you wanted it to have the name like funkychicken.development.domain.com.

But you still wanted your local BIND9 server to be the authority for all *.domain.com addresses.

Firstly create your hosted zone in Amazon Route53

development.domain.com

Find the nameservers that this domain uses (double click on the new entry on the Amazon Route53 panel)

Then find out the IP addresses of each of the name servers associated with your new hosted zone (Amazon use 4 nameservers for redundancy).

Finally

Then here is the magic sauce for your BIND9 configuration…


zone "development.domain.com" {

type forward;

forwarders {
205.251.193.46;
205.251.198.50;
205.251.194.151;
205.251.197.158;
};

};

Result!

This allows all local requests in your company for *.development.domain.com to be sent out to Amazon Route53 nameservers for resolution!

You can test this easily by creating an instance, finding out it’s IP, creating a new A record in Amazon Route53 to point to that A record and ping it from inside your company!

i.e.,

funkychicken.development.domain.com (i-a3b613aa), A record on Route53.

-=david=-

Written by dharrigan

May 31, 2012 at 10:10 am

Posted in life

Java Thread Sleep Tip

Hi,

If you often find yourself writing code similar to this:

...
try {
    Thread.sleep(10 * 1000); // sleep for 10 seconds
} catch(final InterruptedException e) {
    ...
}
...

You can make your intent a lot clearer (and IMHO cleaner) by doing this instead:

...
try {
    Thread.sleep(TimeUnit.SECONDS.toMillis(10)); // sleep for 10 seconds
} catch(final InterruptedException e) {
    ...
}
...

The really nice thing about this approach is that you can substitute the SECONDS to anything else, i.e.,

...
try {
    Thread.sleep(TimeUnit.HOURS.toMillis(10)); // sleep for 10 hours!
} catch(final InterruptedException e) {
    ...
}
...

Neato!

-=david=-

Written by dharrigan

February 8, 2012 at 2:06 pm

Posted in development, java

Tagged with , , , ,

Banning URLs from Varnish using Apache Camel and RabbitMQ – Part 2

Welcome Back!

I hope you found Part 1 on this tutorial useful. You should by now have a running instance of Varnish cache along with a running instance of RabbitMQ. You should also have cloned the Varnish-Ban project from Bitbucket and perhaps had a look through the project structure and source code. I hope there is nothing too unusual in there :-).

In today’s posting we will be covering the following topics:

  • The Varnish-Ban Camel Component
  • Configuring Varnish to respond to HTTP BAN requests.

I hope you enjoy the continuing adventure! :-)

The Varnish-Ban Component

Writing a component to hook into Apache Camel is really quite simple. There are various ways of doing it, but I choose a very expicit and straightforward way to achieve the goal of working with Camel. The main requirements were to:

  1. Create a POJO which implements the Component interface.
  2. Create the Service class that will handle the sending of the BAN request to Varnish.
  3. Add a file called varnish-ban into the folder META-INF/services/org/apache/camel/component. This will allow Camel to auto-register the component.
  4. Create a Camel XML file describing the route and the processing requirements that Camel with respond with.

These steps are described below.

Creating the POJO

Writing the POJO was very simple. Below is a screen shot of the the actual class:

A component in Camel is responsible for creating the Endpoints – in effect it is a Factory.  In my configuration, the Component calls the Endpoint which creates a Producer that invokes the VarnishBanServiceImpl class. I like decoupling of code, so it seemed sensible to me to externalise the actual work of the banning mechanism into a service class that does the work. The service class has the responsibility of sending the BAN request to Varnish. The varnishServerUrl is given to us by Camel when it processes the XML configuration file (see below). The main thing here is we don’t have to do any extra work to obtain the varnishServerUrl – it’s all externalised into the XML file.

Creating the Service Class

The VarnishBanService does all the real work. Fortunately for us, even this class is quite small and very straightforward in its functionality. It simply creates an instance of a HTTP Client (from Apache HTTP Components) and sends off our customised HTTP request (a BAN request) to Varnish:

Our customised HttpRequest – the HttpBan class is very simple:

All that is happening here is that we are extending a base HTTP Class (provided by  HTTP Client) and overriding the getMethod invocation to return our customised HTTP method – cleverly called BAN :-). The toString is a simple helper when we are printing out debug/logging messages. You can create your own particular HTTP Method (SUSHI anyone?) if you have different needs. We could have called our method “LOLBAN” if we wanted to :-)

The remainder of the VarnishBanService class just handles the response back from Varnish and prints out some debug/logging information. Please have a look over to understand how it works. There shouldn’t be any surprises. I’m not handling any exceptions here, but what you could do is wrap up the exception into an AMQP message and shove it back into another Queue for another system to process (a monitoring application for example).

Enabling Auto-Discovery of our Component by Camel

If one creates a file with the same name as our chosen URI (see below in the Camel XML  route configuration section to discover what this is all about), then Camel will automagically register our newly created component and make it ready for use. Like so:

The file has one line in it:

This is all that is required to enable auto discovery in Camel. Pretty neat.

Creating the Camel XML Route Configuration

There are several ways to configure Routes in Camel – one is to use Java DSL to wire things together – another way is to use an XML configuration. I choose to use the XML configuration way just to keep things separate. Underneath the hood, Camel uses Spring, so using an XML configuration file seemed like a nice fit as well.

The file consists of the following elements:

  1. The Source Route. This is our connection to RabbitMQ using the Camel-Spring-AMQP component (see the file applicationContext-beans.xml) in the source code.
  2. What do do when a message comes in (send it on the varnish route)
  3. Splitting the XML payload from RabbitMQ using XPath to obtain the URLs that we wish to BAN
  4. Invoking our Varnish-Ban component against a running varnish instance (http://localhost:6081)
  5. Handling any exceptions that may occur. In this example nothing is done, but we could choose to invoke another Camel component to drop an error message into another queue (banQueueError?)

Configuring Varnish for HTTP BAN Requests

Varnish by default does not permit BANs to occur via HTTP requests. To help encourage Varnish to do so, we need to write a bit of VCL (Varnish Control Language). I’ve put the recipe (a complete VCL file) below (this example is also contained with the conf/varnish/default.vcl file in the Varnish-Ban project):

backend default {
    .host = "127.0.0.1";
    .port = "8080";
}

acl purge {
    "localhost";
}

sub vcl_fetch {
    set beresp.ttl = 5m;
}

sub vcl_recv {
    unset req.http.Cookie;
    if (req.request == "BAN") {
        if (client.ip !~ purge) {
            error 401 "Not allowed";
        }
        ban_url(req.url);
        error 200 "Banned " + req.url;
    }
}

Let’s walk through each section:

backend

This is the backend service that Varnish is fronting – in most cases this will be a webserver. Here I’m instructing Varnish to cache requests from a server running on my local machine and listening on port 8080 (Varnish by default listens on port 6081, so if I hit http://localhost:6081 what will actually be served up is content coming from http://localhost:8080).

acl purge

In this section I’m defining an ACL (Access Control List) list of authorised machines that will be allowed to execute a PURGE (an invented name – I could have called it BANNERS if I wanted to). The ACL is used in the VCL_RECV section.

vcl_fetch

A FETCH is the response from the backend – in the sense that Varnish has “fetched” the response and potentially cached it. Here I’m saying to Varnish to cache all backend responses for 5 minutes.

vcl_recv

A REC(ei)V(e) is the request coming into Varnish from a client. The important things to note here are:

  • I’m removing Cookies. By default Varnish does not cache any requests that contain Cookies.
  • We will do something special if the type of the request (from the HTTP HEADER) is a “BAN” type . I invented this type – it could be called another name.
  • We will only allow those clients defined in our ACL the authority to BAN URLs from Varnish – otherwise we return back a 401 (Not Authorised) to the client.
  • Finally we return a 200 back to the Client once we have finished processing the BAN request.

The example VCL should be put into your “default.vcl” and Varnish restarted. When this is done we are ready to move to the final part of this tutorial!

That’s all for now!

Hopefully by now you will have a running application. In the third and last article of this tutorial we will be sending BAN messages to Varnish and observing the results. Until then, have fun!

-=david=-

Written by dharrigan

January 5, 2012 at 9:23 am

Posted in development, java, software, varnish

Tagged with , , , ,