PCI DSS Shenanigans – Between a ROC and a hard place.

Cha Ching“I’ll gladly sell you one session for the price of two… perhaps three”

I’ve been through audits, I’ve assisted in conducting audits, and I’ve been on the remediation end of audits.  One common theme is that when the quotes start rolling in, the customer usually takes a beat down from the cost of hardware and support.  Now, PCI is larger than just network components, but let’s just focus on some of those parts for now.

As a startup that has been growing and producing more and more credit card transactions, whether it be from online services or in person merchant purchases, has probably been working with bare minimum due diligence designs.  There may not even be an official data center.  As far as web based transactions, you may have some leased servers from company “x” running a LAMP stack, with haproxy/nginx and iptables.   Maybe you are considering leasing space or building a small data center to migrate into with firewalls and load balancers making up part of the new design.  You might as well start preparing now for the inevitable.

The company you bring in to do the audit should not just be there checking off boxes.  They should be doing a pre-audit.   This pre-audit is designed to help you accomplish the remediation aspects or “gaps” between being non compliant and compliancy.  As you are making your way through some of pre-audit comments, you’ll probably see suggestionspci-firewalls-loadbalancers based on the diagram to the right.

I’d like to propose a challenge for those going through an audit or preparing for next year’s fire drill.  Let’s supposed we expect 15 million sessions at any given time flowing through our load balancers.  We have multiple pairs of load balancers in our DC which means we need really large firewalls too.

Have a look at the below terms and definitions as described by the PCI DSS glossary of terms.  Can you come up wih an inexpensive solution for this design?  We’ll talk about traffic flow and whether a network is considered in scope for audit purposes next post.

Hardware and/or software technology that protects network resources from unauthorized access. A firewall permits or denies computer traffic between networks with different security levels based upon a set of rules and other criteria.

Private Network
Network established by an organization that uses private IP address space. Private networks are commonly designed as local area networks. Private network access from public networks should be properly protected with the use of firewalls and routers.

Stateful Inspection
Also called “dynamic packet filtering.” Firewall capability that provides enhanced security by keeping track of the state of network connections. Programmed to distinguish legitimate packets for various connections, only packets matching an established connection will be permitted by the firewall; all others will be rejected.

Network Automation and SDN – Why go through the hassle?

First let me start out by saying that network automation and SDN are not the same.  They share some common goals but achieve them in different ways.


Generic Automation uses the tools that each network vendor thinks we should be working with.  Say OSPF, BGP, ISIS, vlans, etc.  We then apply automation routines against those tools to achieve getting a packet from x to y or placing a server on a given vlan/subnet.


Software Defined Networking (SDN) is a way over used term that is supposed to describe the creation of other non vendor standard tools through a framework of devices that are all connected.  The idea is that we should be able to write our own protocols to achieve what we want, how we want,  when we want, without relying on pre-baked tools from network vendors. 

Unfortunately SDN has turned into simple automation tasks (not because SDN is dead).   CTOs see SDN on the front of a magazine and Network Operations managers want to get rid of the ticket queue.  So it comes down from the CTO’s office, “let’s push SDN”. The operations Manager contracts some coders to write some expect scripts and api calls to accomplish maybe 3 main tasks that are filling up is queue daily.  The CTO then goes to another company with “SDN” under his belt and continues propagating the incorrect term.

Now…  Who really cares?  honestly, its not really that big of a deal.  You could call SDN anything.  Heck, let’s just make up our minds to call it automation. There.. We are done.  It’s settled.  SDN = automation.

One major glitch.  Automation still requires network engineers to troubleshoot and maintain.  SDN does not.  When companies  truly start making the push to SDN, network engineers will either have to change their tool belt or go into management.

Here are some very opinionated fun facts:

Anything not automated is not going to stick around too much longer.

Anyone not keen on automation will most likely find themselves in the same position.

There will be a few that make it through the Network Engineer End of days.

1. The wizard at networking technology X

This guy will be around for troubleshooting and teaching developers how to automate prior to SDN taking a foot hold.  If he doesn’t adjust his tool belt, he will eventually fall.

2. The network engineer willing to start learning to code and automate.

This guy has a much longer life ahead of him.  Like the fortran developers out there today, he will be very handsomely paid.  He will bridge the gap between networking and software development prior to SDN taking over.

Now..  the whole reason why I even started writing this silly post was because I saw this over at Lame Journal.  In that post, John writes

“It seems like virtual networking integration with OpenStack is no longer optional. What’s that mean? What’s happening there?”

I would venture to say that its not just virtual networking.  It’s all networking.  Now that we have a pre-baked API (Neutron) and network vendors are already writing drivers for their hardware, why in the world would anyone write custom automation routines anymore?

The End of Days is coming upon us with a vengeance.

Drivers already supported by Neutron.

Plugin/Driver Name Contact Name IRC Nick
ML2 Neutron Team rkukura, mestery
Big Switch/Floodlight Kevin Benton kevinbenton
Arista Networks Sukhdev Kapur Sukhdev
Embrane Ivar Lazzaro ivar-lazzaro
PLUMgrid Edgar Magana emagana
Mellanox Irena Berezovsky irenab
Cisco Kyle Mestery mestery
Brocade Shiv Haris shivh
Tail-f NCS Luke Gorrie lukego
A10 Networks Micheal Thompson layer427expert
Nicira/VMware Armando Migliaccio armax
Ryu Nachi Ueno nati_ueno
Metaplugin NTT Team nati_ueno, hichihara
NEC Akihiro Motoki amotoki
vArmour Gary Duan garyduan
Midokura Lucas Eznarriaga luqas
Nuage Networks Ronak Shah rms_13
Place Holder Person 1 Name irc_nick

You may notice Juniper is not listed here.  Not to fret, more on Juniper can be found here.

Now, lets say I am a network operations manager.  I get approximately 200 requests a week to add or make some sort of change to a load balancer vip.  I could write my own automation to help with all of these requests.  A10 Networks has had a REST API for a some time.  F5 has been a bit sluggish and offers some services through a rest API if you are bold enough to run their latest code.  But why bother?

You’ll notice A10 is listed above as having drivers for Neutron.  (I have a team mate in the processes of vetting these drivers. I will write more when I have more info)

So whether we have made the jump to openstack or not, it would seem we may be able to make use of Neutron in any environment.  Time to get the virtual lab setup.  For this, I will request a demo vm from A10 Networks.

Additional Credits:

Awesome pic of a neutron taken from this post.

Microsoft OneNote for Mac OSX

Finally.  A full featured “note” client for Mac OSXonenote-logo

From Microsoft:

“Today we announced three major OneNote developments, including the first version of OneNote for Mac. We are very excited and proud to deliver this to you today.

We’ve seen the countless requests for a Mac client of OneNote, and we’ve been hard at work to deliver it. We’ve been counting the days to finally share with you that OneNote for Mac is now available and you can download it from the App Store for free today!”

It’s about time.  I can only assume the prevalence of Evernote across all platforms scared Microsoft into releasing OneNote for Mac OSX.  This was one of my major pain points when I was given a Mac at work.

More can be found on the Microsoft Blog.

Juniper introduces more Network Deception tools

Well, anyone familiar with Juniper’s acquisition of the Mykonos Web-centric Intrusion Deception products may or may not have rolled their eyes at it.  What the heck was Juniper thinking? They were buying a product that didn’t really have an established market and was mostly  mis-identified and compared with Web Application Firewalls, which resulting in losing by features comparisons to those products almost everytime (because it is NOT a WAF, mind you).  Well, my thoughts on it was that Juniper didn’t buy the product for the product itself, but to use as a spring board for launching a threat-intelligence solution, which came to fruition with the Spotlight Secure product. Spotlight Secure was interesting because it could take information multiple sources, such as: Juniper WebApp Secure (the new name for the Mykonos product line), SRX Security gateways, and the Juniper DDOS Secure product line.  While that may be cool for customer to have their devices talking among each other, the big advantage is when multiple global customers have their Spotlight installations talking to the cloud and then provide threat information to other customers. Pretty clever Juniper….but wait there’s more.

On Feb 25, 2014, Juniper announced a new product: Juniper Argon Secure, an add-on subscription that can be leveraged on Juniper SRX Security Gateways. While I’m still investigating what all the product can do, it appears that it will do more in the way of the network fake-outs (similar to what Web AppSecure provides with its TarTrap mechanisms) such as fake fileshares which detect when malware reaches out to network shares attempting to gain access to confidential data, or to latch its hooks further in the network.  Although none of the literature specifically stated that it would have faux-SMTP processes, I would think that would be par for the course as well, since the press release stated the product can detect applications that “attempt to send data outside the company network”.

Having productized honey-pot/tar-pit type products that can automate their detection and leverage that information to protect other parts of the network by enabling on-demand firewall policies, possibly quarantining hosts to specific network segments, and sharing zero-day detection data with other Juniper customers seems like a step in the right direction. This might just change the outlook for Next-Gen Infrastructure Security…


BTW, if you think the new stuff is cool, you should check out the Secure Access/Access Control Gateways and Juniper Secure Analytics (formerly STRM) integration with IF-MAP – a powerful shared information repository in it’s own-right.

World War III is already upon us and it’s not being fought with guns

juniper-logoAs I watched the Juniper keynote from RSA 2014 (Given by Nawaf Bitar), I couldn’t help but to sit back and just think for a bit.  Wow.  This guys is right.  We have been complacent and are becoming more and more complacent with our security.  I think it comes back to some very basic human nature.  What we think is normal is usually what we are surrounded by.  An alcoholic may think it is perfectly normal to be at a bar drinking because everyone he hangs out with is also at that same bar.  Step outside the bar though and we find that 90% of the nation’s alcohol consumption comes from 30% of the people.  This leaves the other 70% of the nation to consume the remaining 10% of the booze.  It’s clearly not normal to be in a bar drinking.  I digress.  Lets get back to the topic at hand.  What is normal?  Is it normal to accept friend requests from people you have never met?  (rhetorical question)  Let’s go back to the 1920′s for a sec.  It was perfectly normal to NOT use curse words in public.  It was normal for men to wear suits and ties out in the FIELDS.  It was normal for women to wear coverups while they were IN the pool… Let’s fast forward a bit.  Over the years we have slowly allowed more skin to be shown, more curse words are said in public.  Ties have been replaced by “business casual” or less.  More violent video games, etc etc…  And the more we surround ourselves with this, the more it becomes normal to stretch the boundaries to constantly create a new normal which is less that our original which continues to stretch the boundaries.  Now…  Let’s say it.  It’s normal to be careless with our security.  There may be a 30% number out there that are very security centric but frankly, the other 70% are not.  We do in fact accept friend requests from people we don’t know.  We do use open source software that we really don’t do proper inspection on.  We don’t insist deep security dives on closed source software.  How many of us actually verify the md5 checksums when we download software?  We have been opening our doors a little more at a time.  A little more bad stuff gets in every day which becomes the new norm.

Now enough of that.  Let’s talk about the Keynote.  I enjoyed watching it.  It was a little weird to not have a product pitched behind it for the world to see…. maybe that was by design.

By all means, please watch this Juniper keynote by Nawaf Bitar.

I am looking forward to seeing product demos. Browsing Juniper’s website, I see a blurb on Juniper’s Spotlight Secure.

“Once an attacker is identified and fingerprinted on a subscriber network using WebApp Secure, the new service will immediately share the profiles with other subscribers, providing advanced real-time security solution across multiple networks.

Spotlight Secure will put non-IP-based attacker profiling at the center of a framework that gathers and distributes attacker fingerprints to a worldwide network of inline security solutions.”


Important security patch for Mac OS X

Just updated to Mac OS X 10.9.2.  I urge any of you running Mac’s to update ASAP.

“Apple has released OS X 10.9.2 which, you’ll be delighted to know, improves the “accuracy” of the unread message count in Mail, and fixes the autofill feature in Safari among other little tweaks.

It also just so happens to snap shut a gaping security vulnerability that potentially allowed hackers to hijack users’ bank accounts, read their email, steal their passwords, and compromise other SSL-encrypted communications.”

More can be found at the Register

Image taken from valuewalk.com

Nokia and Juniper?

Nokia Sign

Just ran across this on Reuters:

“(Reuters) – Finland’s Nokia is considering buying U.S.-based Juniper Networks to merge into its telecommunications network gear business, German’s Manager Magazin Online reported, citing unidentified sources.”

More of the article can be read here.

I am always leery of articles that “cite unidentified sources”.  Especially with all the shenanigans going on right now with Juniper and the elliot group.

“Jan 13 (Reuters) – Hedge fund Elliott Management Corp urged network equipment maker Juniper Networks Inc to buy back shares, start paying a dividend and consider slimming down, less than a week after it offered to buy Juniper’s rival, Riverbed Technology.”

More on that here.

Of course none of us really ever know what is going on do we?  :)

Passion for Pushing Packets Pretty Please

All those in favor?

The EtherNet
The EtherNet

All those against?

You’ve seen them… the peeps that are in IT because they thought it would pay well…. not a bad career move…  because it does pay well.  What they didn’t bargain for was the data center outages, sleepless nights, and the forever annoying cell phone that now connects our brains to the ether.

By the way.

What do you use to catch an EtherBunny?

Survey says…

Why, an EtherNet of course….

If you chose not to laugh at that, you may want to check out this post by John over at Lame Journal

Give me a use case for this and win a $5000 giftcard (IRDP)

Taken from the ScreenOS help guide:

ICMP Router Discovery Configuration

ICMP Router Discovery Protocol (IRDP) is an ICMP message exchange between a host and a router. The security device is the router and advertises the IP address of a specified interface periodically or on-demand. If the host is configured to listen, you can configure the security device to send periodic advertisements. If the host explicitly sends router solicitations, you can configure the security device to respond on demand.

*Just kidding on the $5,000 gift card :) *

ScreenOS supports IRDP on a per-interface basis. You must assign an IP address before IRDP becomes available on that interface. By default, this feature is disabled. You can configure this feature in a high availability (HA) environment with NetScreen Redundancy Protocol (NSRP).

To enable IRDP on an interface, select the IRDP Enable checkbox. Enabling IRDP initiates an immediate advertisement to the network.

(Primary IP) is the primary IP address of the security device. Select the Advertise checkbox and enter a preference status for the security device. The preference status is a number from -1 through 2147483647. Higher numbers have greater preference.

To configure IRDP options, enter the following, then click Apply:

Broadcast-address: Select this option to broadcast the advertisements. The default address is (all hosts on the network).

Max Advertise Interval: Maximum number of seconds that pass between ICMP advertisements. The default value is 600 seconds. The valid value range is from 4 through 1800 seconds.

Lifetime: Number of seconds for the lifetime of the advertisement. By default, the lifetime value is three times Max Advertise Interval. You can also set the lifetime value. The valid value range is the Max Advertise Interval through 9000 seconds.

Min Advertise Interval: Lower limit (in seconds) of the advertisement period, which is calculated to be 75 percent of the value entered for Max Advertise Interval. You can also  specify a number from 3 through the Max Advertise Interval value. When you change the Max Advertise Interval value, the Min Advertise Interval value is automatically calculated.

Response Delay: Number of seconds to delay response to a client solicitation. By default, the security device waits for 0 to 2 seconds before responding to a client solicitation request. You can change the default setting from no delay (0 seconds) to up to a 4-second response delay.

Init Advertise Interval: Number of seconds during the IRDP startup period allocated for advertisement. The valid value range is from 1 through 32 seconds. By default, this period is 16 seconds.

Init Advertise Packet: By default, the security device sends out three advertisements during the specified startup period (Init Advertise Interval). The valid value range for the number of advertisements sent is from 1 through 5.

To disable IRDP, deselect the Enable checkbox. Disabling this feature causes all IRDP-related memory for this interface to be removed.

By reading this post, you agree to not actually submit a request for a $5,000 gift card.  If any such request is made or any law suites filed, you will be rewarded with $5,000 worth of toe jam.

Crisco May Lead to Network Obesity – But is overweight so bad?

We just finished up a proof of concept with the good ole boys in green. Yeah I said it… Cisco

Luxurious Cisco, no wonder they are so expensive
Luxurious Cisco Hotel and Spa…..ehhh.. maybe not.

Have I mentioned how proud Cisco is of their gear? I won’t mention any pricing here but WOW.  Not exactly stuff you would find down at Shirley’s discount match and gasoline factory.

So the PoC went a little like this:

  1. Two distinct regions or availability zones
  2. Each region’s security devices were in an HA cluster
  3. Each HA cluster could completely fail and the other region could pick up the slack (ala BGP, wait… Cisco firewalls don’t support BGP)
  4. Load balancing was shared across both regions equally (thanks again BGP)
  5. Each region could easily scale to a more than reasonable size without the use of spanning tree or virtual chassis
  6. Each region’s kit could be upgraded with no noticeable downtime
  7. And entire region could fail without the services behind it taking a hit.
  8. Each region could support 160Gb throughput to the other region and out to the WAN if need be.

Hardware included in the Proof of Concept

  1. Cisco Nexus 6001
  2. Cisco Nexus 6004
  3. Cisco ASR 9001
  4. Cisco ASR 9006
  5. Cisco ASR 9010
  6. Cisco ASR 1000 (used as a “firewall” because Cisco does not have a firewall that can participate in BGP) Our Design Calls for a Juniper SRX.
  7. A10 TH5430
  8. Spirent test boxes with a ton of 10Gb interfaces

Anyone care for the results?

Network Tips, Tricks, and General Shenanigans