Oct 31 2015

Happy Halloween

Happy Halloween

Permanent link to this article: http://ericsilva.org/2015/10/31/happy-halloween/

Oct 29 2015

Update all Python pip packages

pip freeze --local | grep -v '^\-e' | cut -d = -f 1 | xargs pip install -U

Permanent link to this article: http://ericsilva.org/2015/10/29/update-all-python-pip-packages/

Sep 30 2015

Wood Badge N6-528-15

Wood Badge N6-528-15

Permanent link to this article: http://ericsilva.org/2015/09/30/wood-badge-n6-528-15/

Sep 01 2015

Old School

Old School

Permanent link to this article: http://ericsilva.org/2015/09/01/old-school/

Aug 17 2015

MuleSoft CloudHub to Amazon AWS RedShift Proxy

I encountered a situation in which we needed to connect a Mule ESB flow running in MuleSoft’s CloudHub to an Amazon AWS Redshift cluster.  This doesn’t work out of the box.  I’ve outlined the steps to create a proxy server below. The original write up I did is also available in this Gist.


Below are the steps needed to create and configure a proxy between MuleSoft CloudHub and Amazon AWS Redshift using an Amazon AWS EC2 instance.

The Problem

EC2 and Redshift instances are configured to support jumbo frames (MTU for ethernet interfaces is 9001). However, some routers between endpoints have a standard Ethernet MTU size (1500), which causes an inability to communicate with announced TCP MSS size (8961). The reason for this issue is that the PATH MTU discovery process relies on ICMP, specifically Type 3 Code 4 / Fragmentation Needed, and currently on Redshift ALL ICMP traffic is denied (regardless of Security Group configuration).

MuleSoft CloudHub uses the standard ethernet MTU (1500), and cannot connect to a Redshift cluster by default. The steps below document how to create a lightweight IP proxy using an EC2 instance.

Configuration Details

  1. Create an AWS instance in the same Availability Zone (AZ) as the Redshift cluster using the following criteria:
    1. AMI: Ubuntu Server 14.04 LTS (HVM), SSD Volume Type - ami-d05e75b8 (or similar)
    2. Instance Type: t2.micro
      • initial performance tests have shown this to be adequate as the proxy is not CPU/RAM intensive.
    3. Instance Details: accept default or modify depending on VPC configuration
    4. Tag Instance: cloudhub-redshift-proxy
    5. Configure Security Group:
      1. Restrict SSH access to trusted IP Ranges
      2. Add Custom TCP Rule(s) for each Static CloudHub IP which will access the Redshift cluster
        • Protocol: TCP
        • Port Range: 5439 (default Redshift port)
        • Custom IP (using CIDR notation): x.x.x.x/32 (e.g.
  2. Launch instance, and choose an existing SSH key pair that will allow you to SSH to the instance.
  3. Disable Source/Destination Check
    1. Select the instance from the EC2 Instances list
    2. Select Actions > Networking > Change Source/Dest. Check
    3. Click the Yes, Disable button
  4. Once instance is launched, connect to the instance using the Public DNS/IP:ssh ubuntu@server.eu-west-1.compute.amazonaws.comssh ubuntu@
  5. Enable IP packet forwarding
    1. Open the /etc/sysctl.conf in vi or vim:# sudo vi /etc/sysctl.conf
    2. Uncomment the following line:net.ipv4.ip_forward = 1
    3. Save the file
    4. Apply the changes with the following command:# sudo sysctl -p
  6. Apply iptables rules for TCP MSS adjustment (assuming using the default Redshift port 5439) a. Enter the following two commands:sudo iptables -A PREROUTING -t mangle -p tcp --sport 5439 --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1460sudo iptables -A PREROUTING -t mangle -p tcp --dport 5439 --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1460
  7. Make NAT translation from “proxy” to a Redshift Cluster IP (RS_IP is the cluster IP address,LOCAL_IP is IP address for eth0 interface of “proxy” host)
    1. You will need the private IP of the EC2 Proxy instance. You can find this by looking at the AWS instance details or by typing ifconfig at the command line within your SSH session and look at the eth0 device.
    2. You will need the IP of the Redshift cluster as well.
    3. Enter the following command replacing RS_IP with the cluster IP, and LOCAL_IP with the EC2 private/local IP for eth0:sudo iptables -t nat -A PREROUTING -p tcp -d LOCAL_IP --dport 5439 -j DNAT --to-destination RS_IPExample:

      sudo iptables -t nat -A PREROUTING -p tcp -d --dport 5439 -j DNAT --to-destination

    4. Enter the following command:sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
  8. Logout of the SSH session. The proxy is configured.

Permanent link to this article: http://ericsilva.org/2015/08/17/mulesoft-cloudhub-to-amazon-aws-redshift-proxy/

Jun 23 2015

Obscured by Clouds

Obscured by Clouds

Permanent link to this article: http://ericsilva.org/2015/06/23/obscured-by-clouds-2/

Jun 20 2015

Interesting Caterpillar

Interesting Caterpillar

Permanent link to this article: http://ericsilva.org/2015/06/20/interesting-caterpillar/

Jun 20 2015

Nice day at Hunsicker’s Grove

Nice day at Hunsicker&;s Grove

Permanent link to this article: http://ericsilva.org/2015/06/20/nice-day-at-hunsickers-grove/

Jun 06 2015

Gorgeous day outside

Gorgeous day outside #nature

Permanent link to this article: http://ericsilva.org/2015/06/06/gorgeous-day-outside/

Jun 06 2015

Hello Mr. Snapping Turtle

Hello Mr. Snapping Turtle #nature

Permanent link to this article: http://ericsilva.org/2015/06/06/hello-mr-snapping-turtle/

Older posts «

Fetch more items