Category Archives: Business Agility

IT Service Brokerage: Technical Mindset – Infrastructure

“…if an application moves from an environment where disks/volumes are mounted using WWNNs/WWPNs as end-point IDs (fibre channel) to an environment with IQNs as end-point IDs (iSCSI) we often have to re-validate and re-engineer.  If the application were to list its own requirements it would actually just be something like ‘xGB block storage, isolated, with <performance guarantee 1> and <performance guarantee 2>’.  There would be no mention of WWPNs or IQNs, fibre channel or iSCSI.  The list above is the type of [automated/attached] description needed that would help make the application portable.  It fits into a trust-based model (aka Promise Theory)…”

The above is an excerpt from my recent post on our Cisco UK & Ireland blog.  You can read the full post here: http://gblogs.cisco.com/uki/it-service-brokerage-technical-mindset-infrastructure/

 

DevOps Collections #4

The plan is to produce some more guides/write-ups over the coming weeks… but for now, here’s the latest Continuous X and DevOps related noise:

1_DevOpsBox

Source CY14 W39–44

 

DevOps Collections #3

Another aggregation of Continuous X and DevOps news and information put together for colleagues at work but thought to be worth sharing outside of that group.  * = help with prioritising what to read if you’re hard pressed for time.

  • Where does DevOps work in the enterprise? (itskeptic.org) *
    • The terms “Bimodal IT” and “Trimodal IT” have recently been used to describe the potential split of strategies for different classifications of IT systems – i.e. ‘old stuff’ and ‘new stuff’ and the different opportunities with each (pretty much = don’t hold up ‘new stuff’ because of ‘old stuff’ requirements – they’re different).  This article talks about ‘DevOps in Enterprise’ being more successful when when addressing the ‘new stuff’ rather than being reverse engineered against existing/legacy systems.
  • Making DevOps Work in Complex Enterprise Environments (devops.com)
    • However, if you are going to try to reverse engineer against the ‘old stuff’ to some degree then here’s a starting point.  Interestingly, we have reference to where data centre tech/infrastructure services can make a different in a more generic sense: “Save that bespoke Windows config as a virtual machine, then you can subsequently redeploy it… use an enterprise infrastructure-as-a-service (IaaS), whether private or public cloud, hosted or on-premises, especially for newer applications… adopt platform-as-a-service (PaaS), either out-of-the-box or by building a PaaS that will support your unique environments”
  • DevOps with a purpose: It’s about your applications (devops.com) *
    • “I recommend analyzing your business goals and application portfolio using the pace-layering method, which focuses on three categories of applications – systems of record, systems of differentiation, and systems of innovation”.  Back to the ‘old stuff’ vs. ‘new stuff’ but with a bigger link to systems thinking and what can really affect the business – I like these classifications.
  • Docker closes C round for $40m (DevOps.com)
    • “There have now been 21 million downloads of the Docker platform, up from 3 million at DockerCon. Over 35,000 “Dockerized” applications are now available on the Docker Hub Registry, and more than 13,000 Docker-related projects have launched on GitHub. Docker has also seen rapid growth in the technology partner ecosystem with over 100 companies – including industry heavyweights Amazon, Google, IBM, Microsoft, Red Hat and VMware – having announced Docker–supporting platform initiatives.”
  • The DevOps Mindset: Real-World Insights From Tech Leaders (Rackspace – Click “Download Now”)
    • Collaborative culture, elevated & shared goals, collaborative culture, elevated & shared goals, collaborative culture… if people say it enough it will happen everywhere right?  Some interesting comments from ‘DevOps leaders’ within different orgs in this document.  The real question that still remains is what changes in the DevOps model in order to apply it within a large multi-national organisation? You may notice that none of the interviewees sit within an org bigger than 2000, most are in the hundreds… the question still remains.
  • Microservices and PaaS (activestate.com) *
    • An interesting overview of complementary application architecture changes – small distributed units/modules instead of big monolithic tiers.  A 3 part article.
  • Change your thinking about Change Management (devops.com – Chris Riley)
    • “change management, the tools, can be detached from change management the governance. And then no longer tied to the reactive philosophy”.  A practical look at opportunities around Change Management within a DevOps philosophy.
  • Putting the Ops in DevOps (indecorous.com)
    • “the nearest I can come to “you’re doing it wrong” is when people announce the death of Ops as a job function…. in truth it’s very, very hard to excel in both the development and operational aspects of building and running a large computer system. These people do exist, but trying to find them and hire them is hard (and expensive). Moreover, beyond a certain (small) scale it’s actually woefully inefficient to have everyone doing everything.”  Tips from someone ‘doing DevOps’.
  • To Help DevOps-ify The World, ScriptRock Raises $8.7M (Forbes)
    • ”the company’s product, GuardRail, focuses on helping IT departments understand how their various systems are operating wherever they may be located. Instead of this information being strewn across the organization in multiple departments and locations, GuardRail brings it all to one central location”.
  • How to skill up in DevOps and what rates you can expect in 2014-15 (eSynergy)
    • An insight into the kind of rates contractors can expect for a ‘DevOps’ skillset.
  • Developing and enabling a DevOps culture in your team (devops.com) *
    • An interesting take on how to revitalise existing personnel under a DevOps effort instead of seeing them as a barrier to change.
      • Additional anecdotal points elsewhere: theopsmgr
  • Dissecting Effective Developer Workflows to Save Time & Lower Costs (Flux7)
    • A look at the realities and results of ‘shifting things left’.
  • Harvard Business Review Survey: IT responsiveness predicts business success (devops.com)
    • “Nearly half of respondents reported that their companies have missed opportunities because their IT department was too slow to respond to shifting business needs. And approximately 49 percent reported that a lack of bandwidth in IT is the primary obstacle of the organization to take advantage of digital technologies… Approximately 42 percent of organizations report that they have a poor track record of facilitating collaboration across IT and business functions.”
  • The Road to DevOps (ActiveState)
    • An example of the dangerous mindset out there that can cause ‘Ops’ to go against the grain of a DevOps movement.  This article is not the guidance needed for establishing the right culture imo.
  • Q & A: Puppet Labs CIO Nigel Kersten (devops.com)
  • Leading the Horses to Drink … Support and Initiate a devops transformation – Damon Edwards (video)

FYI The DevOps Cookbook is reportedly at draft/review stage.  This will likely turn out to be a ‘must read’.

There is also apparently ‘DevOps Training’ for ‘DevOps Certifications’ popping up now.  While it helps with the foundational knowledge side of things it is also an indication of the beginnings of a cottage industry that might ultimately break the DevOps ‘following’.  With that in mind, two more articles:

 

DevOps Collections #2

Another inch up the curve… As before, there are notes so that you can be selective with your reading and there are * against the reads to maybe prioritise if you’re hard pressed for time.

  • Enterprise DevOps – The Opportunity & The Challenge (Benjamin Wootton) -> A concise slide deck that could act as a guide to your own talking points if facilitating an ‘Intro to DevOps’ session.
  • Common Objections to DevOps from Enterprise Operations (Dev2Ops) -> “The quickened pace puts a lot more pressure on the centralized ops team because they often get the work late in the project cycle (i.e. when it’s time to release to production). Because of time pressure or because they are over worked, operations teams have difficulty turning requested work around and begin to hear developers want to do things for themselves….. unfortunately, a centralized devops team can become a silo and suffer from the same “late in the cycle” handoff challenges the traditional ops group sees.  -> A practical view of the kind of situation ‘Agile‘ can create on the Ops side when it’s mainly Dev pushing a cultural shift towards DevOps. *
  • Include Ops people in user stories for Operability (HighOps) -> “Many organisations use terminology which is unhelpful and counter-productive towards Operability
  • The Continuous Delivery Maturity Model (InfoQ) -> Taking five key categories, Culture & Organisation, Design & Architecture, Build & Deploy, Test & Verification and Information & Reporting, the aim of this article is to establish a method of measurement for where an organisation is with ‘Continuous X’.  This particular model defines five maturity levels: base, beginner, intermediate, advanced and expert.  Measuring against this model can help you to work out how best to support your customer.  Image available. *
  • Two Ways DevOps Unlocks the True Potential of Agile (devops.com) -> “The good news is, organizations can close their gaps and realize the true value of Agile development by incorporating DevOps processes and tools into their systems development life cycle (SDLC)… The DevOps principle to employ here is getting the operations team involved in testing earlier in the SDLC – essentially shifting their work left – and the tool that can help is service virtualisation”.  This article is effectively saying if Ops are involved earlier in the SDLC then they can help get QA/testing environments as close to a real-world Production environment as possible and monitor the whole SDLC.  This is advantageous when considering the implications of Continuous Delivery and Deployment. By simulating constrained or unavailable systems across the SDLC, service virtualization enables developers, testers and performance teams to work in parallel for faster delivery and higher application quality and reliability
  • DevOps and Change Control (sdncentral) -> A rhetoric and some guidance on ‘if everything’s constantly changing, how do we manage and track the change?’
  • Why DevOps Matters To The CIO (contino) -> “technology that can be used to innovate and succeed in a competitive marketplace… the challenge of delivering technology that enables their business to succeed… DevOps is a movement and set of best practices that has a lot of value for the CIO who is operating against this backdrop.”  Effectively a DevOps sales pitch to a CIO. *
  • How to Celebrate DevOps Success (ranger4) -> “So what might a DevOps SMART goal look like?” Metrics and rewards acting as a stimulus for successful DevOps…?  Status, Certainty, Autonomy, Relatedness and Fairness (SCARF) needs to be looked at in each person’s role/group of people in the eyes of this writer.
  • DevOps: The Operational Amplifier (DevOps Journal) -> “Maintenance windows exist in the enterprise, after all, to manage expectations with respect to downtime and disruption specifically because of the interdependent nature of enterprise applications…..The thing is, these numbers are only going to get worse as the Internet of Things continues to put pressure on organisations”  This article takes a look at the unaligned growth of IT services vs. the people operating said services.  DevOps can be the tool, the ‘amplifier’ to allow a few Ops folks to manage a world of [near] exponential growth.  “That means APIs – strong APIs – as well as extensibility and flexibility. Infrastructure cannot remain rigid and static in an environment that is rapidly changing. It must be dynamically configured, extensible, and imminently flexible.” *
  • Testing in a Continuous Delivery world (SD Times – Rob Marvin) -> “If there’s one overarching principle, it’s to automate everything”  This article gives a good indication of the anticipated push of automation through the entire stack from a SDLC perspective.  We at Cisco talk about automating at an infrastructure level but tying full software revision control inclusive of infrastructure services is where things are heading… i.e. Regression testing is mentioned in this article – if I regress my software, I’d like to regress the infrastructure state below it… in an automated fashion…

DevOps Collections #1

DevOps continues to heave itself up the hype bell curve and with that there are some very interesting reads out there.  I put the list below together for my team at work, it’s a summary of recent articles that touch on different sub-areas of DevOps.  I’ve added notes so that you can be selective with your reading.  I’ve also put * against the reads to maybe prioritise if you’re hard pressed for time.

If you wish to read something that takes you on a bit of a ‘business view, real world’ journey through ‘Why DevOps, what does it deliver, etc.’, I can recommend The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win.  It’s actually a candidate for bedtime reading without the person (i.e. me on this occasion) telling you so with a smirk on their face!

Give me what you’ve got!

Application Centric Infrastructure (ACI) gets me as giddy as UCS did, here’s a great demo video very recently published:

 

We’ll be looking at ACI from a number of different angles over the coming months…

Capturing DevOps – Part 2 of 2

In Part 1 of this update I briefly ran through the Cisco goUCS tool and how it can be used as a means to an end with regards to capturing XML passed between the UCS Manager java client and UCS’ main XML API.

In this ‘Part 2’ I’d like to push on right into the world of programming against open infrastructure systems.  Along the lines of explanations in previous posts, I mean ‘open’ in respect to how you interface and communicate with the systems to form provisioning and monitoring actions.

This shift comes under the umbrella of the remit of DevOps.  Like many terms used in IT, there’s a question mark over whether the term has an ‘etched in stone’ definition today; the full definition will take shape over time.  It is well enough defined to be able to identify meaningful technologies and trends that it relies upon however.  DevOps is essentially a response to the growing awareness of the disconnect, or “Wall of confusion” as it has been called, between software development and the systems supporting the running of said software applications.  I like to compare it to Systems Thinking and how you would pragmatically optimise the delivery of Business Applications without considering current-day people (i.e. structure and skills) and toolsets to a point where they become inhibitors to a given business’ ICT ideology.

Note. I'm not a programmer.  A lot of this stuff is quite new to me so I'm explaining things from that view point. i.e. somebody near the far end of the 'Ops' side of the DevOps movement.

Over the coming years you can expect to see efforts to address the conflicting skills, backgrounds, motives, processes and tools associated with the two different starting points to delivering a software application.  “Agile Software Development” initiatives in many business sectors and verticals could be one of the major stimulants for speedy change in the way things are done.

If you’ve not got a programming background the app that we’ll be getting onto soon could definitely jump out at you in quite some gory detail that looks quite frightening at first!  I took a first look at it and thought ‘hmmm, not for me’ anyway…  Please stick with it though 😉

Here is what we essentially have today:

StackComm

Here is where the app that I’m introducing you to sits… API-calls downstream in all cases, SME input at the app level (previously the ‘Speakers’ each time something needs doing):

GeneralAPITool Placement

Sorry about the hazed text on the left – that’s a ‘save to image’ thing… those tags are vNet (virtual network), pNet (physical network) and vService (virtual network service e.g. firewall) -> they merge together to create network containers.

As you can see, the app that’s identified in the diagram hasn’t got a fancy name, and before you see it I’ll state that it also doesn’t have a massively intuitive interface if you’re not a SW programmer.  It was written by a colleague of mine, a chap called Rob, who does some programming as a hobby… that’s right, a hobby… tut! why can’t I be like that!?

The app can be found here: https://github.com/ciscodev/GeneralAPITool.  There is a “Download Zip” link on the right hand side of the github page.

Rob’s aim was to create a straightforward API-calling SDK-type tool relevant to the infrastructure elements that we, in the Cisco UKI DC Team, deal with day-to-day rather than doing everything ad-hoc and specific to a given solution or product.  UCS Director includes a lot of what we’re doing in a much nicer interface but we wanted to see under the hood a bit more from a programmability perspective.  It came about after we ran a “Lunch & Learn” session on “Programmability” and we wanted to automate the login and delivery of XML to Cisco Prime Network Services Controller during that session.  Rob’s bigger idea was to develop the tool further to be able to login to many systems and have some of the needed XML already catalogued.  Hours of ‘hobby time’ both developing the app and cataloguing XML from what I can see!  The app that he’s posted on github has a structure and catalogue to run against Cisco UCS but it can be expanded to support other systems easily enough (Rob has a newer version that he’s working with…).  All of the systems that we work with have an API guide available.  After the app has been downloaded it needs to be unzipped onto a machine running Python… because it’s written in Python… with these modules available (default modules):

Side Note.  It's worth mentioning that the path of least resistance is probably to clone/create/spawn a Linux-distro VM and run from there.  It works fine on a Mac, Windows just might cause you to be looking at things that are unassociated with the app...

 

I’ve unzipped, what have I got in front of me?

Once unzipped, the application has 3 levels of directory and file structure, these directories present a hierarchy of the application, the application’s modules/classes etc. and some pre-built XML files.

Here’s the layout:

File Structure

The top-level files are where you’ll spend most of your time.  Under the “data” directory, the directories then get into the particular system that we’re sending requests to.  That’s “UCS” alone in this case.  The XML files have been catalogued as part of the app in the “b” directory (‘b’ for B-Series UCS).  This is basically what we were doing in Part 1 but many XML strings have been captured instead of the single one that we did.  The classes reference each of the XML files and provide the context needed to deliver them + log.  Some of the XML files are targeting specific areas of the UCS system.  Those XML files will be indexed in the relevant class… e.g. “network-MAC-pools.xml” is for MAC pools stored by the system so they sit in the “network.py” class as a re-usable ‘module’.  This modularisation is the right way to write an app such as this.

To use the app it’s wise to use an Integrated Development Environment (IDE).  I’m using Sublime Text, you can use whatever you feel most comfortable with.  If we open the “main.py” file within the IDE you’ll see a starting point of the application – there’s a lot in there!  There are dependencies across the app as you move through the classes (which are like re-usable modules) and XML files.  The relations break down as the following:

GeneralAPITool Dependencies

Each of the arrows above has a pointer aimed at an inheritance statement (identified as an “import” line).  The inheritance of a given function is basically in the opposite direction to the pointer; the class is ‘sucking in’ what it’s pointing at.  If we were to add a new system, such as Prime Network Services Controller, Prime Data Center Network Manager, Nexus 1000v (VSM), etc. we would add directories under data folder in line with that system.  E.g. Nexus could be ‘/data/nexus/classes/’ + ‘/data/nexus/xml/’ with classes and XML broken down as appropriate.  The new system would then be imported and added into a new class using main.py as a guide.

What do all these imports give each of the modules?

Let’s walk through some of the diagram briefly…  All of the purple modules are Python modules and they each do a particular pre-built ‘standardised’ function that some of the newly written classes reference.  e.g. urllib2 offers a pre-built way to interface with a HTTP-based API (i.e. REST or SOAP) as it’s a library for opening URLs.  The hyperlinks listed further up in this post provide extra details on those default modules.  We then move onto the application’s classes written for this app.  The “main” class is where you drive the app from (in this case it has code populated for UCS).  It has an ability to log including error logs so that explains the arrows to the appropriate module + class.  For “main” to drive things it relies upon an understanding of the managed system.  The import of “ucsclass” gives it that logic for UCS.  “ucsclass” then in-turn pulls on the logic of more specific ‘UCS sub-function’ classes; hierarchical and modular in a logical sense.  It also logs errors…  Lastly, the only other newly written classes unaccounted for are “myFunctions” and “XMLFunctions”.  “myFunctions” is the engine that delivers XML to a system.  It makes use of “XMLFunctions” which is a class used to load, capture and manipulate XML.  Manipulate?  We need to manipulate XML because of the authentication mechanisms built into the way many APIs work.  In the case of UCS a successful login gets a CookieID sent back in the response from the UCS system.  The same CookieID then has to be present in every other request to the system otherwise it seen as an unauthenticated session.  “XMLFunctions” does the relevant parsing and editing to make that work.  Both “myFunctions” and “XMLFunctions” also need to log errors…

 

How do I use the app?

As a first step, I would suggest that you quickly update the address and authentication details of your UCS instance.  The UCS instance could be a real system or an instance of the Cisco UCS Emulator.  This edit needs to be done in “main.py”:

UCSInfoWe now have a valid starting point for the application as far as managing UCS is concerned.  What’s all of this endless code in main.py though?  Well, it’s actually ‘everything you would ever want to do with UCS’ and the reality is that you may only want to do a subset of what’s been coded.  So, the second step is actually to make a copy of main.py and give the new file any name that you want.

Once the copy of the file has been opened you’ll want to know which bits should remain fixed and which bits are for you to play with.  Everything up to here needs to stay:

Do Work After Lines

Anything beyond those lines is where you can test using a python app to read and write against UCS using its API!

 

How do I use it… practical examples:

Hmmm, let’s scroll down this mind boggling text… and… yep, how about we spread a few things across a few files…  Here’s the flow:

  • File 1 = Get some information about the UCS Fabric Interconnects and print it to the console.
  • File 2 = Print the VLAN DB to the console -> Create a new VLAN -> Print the VLAN DB to the console.
  • File 3 = Print the org list to the console -> Create a new org -> Print the org list to the console.
  • File 4 = Remove the previously created VLAN -> Remove the previously created org.

File 1:

  1. Keep “getFIDeviceInfo”, “getFISystemStats” and “getFIFirmwareVersions” def sections below the lines highlighted above and save a new file.
  2. Edit the list right at the bottom of the file to change what information is fed back to us (i.e. a main.’def’ line to ‘print’ each of the items above.  So, that’s “Main.getFIDeviceInfo”, “Main.getFISystemStats” and “Main.getFIFirmwareVersions”).
  3. Correct a typo that I’ve noticed under “def getFIDeviceInfo”: “blades = self.ucs.getFIDeviceInfo()” should actually be “blades = self.ucs.fi.getFIDeviceInfo()” as it needs to reach down to the “fi” class.  I’ll mention that one to Rob!
  4. Run the file.

An output:

File1

File 2:

  1. Keep “getAllVLANs” and “createVLAN” sections below the lines highlighted above and save a new file.
  2. Re-order and edit so that there is a copy of “getAllVLANs”, then the “createVLAN” def and then another copy of “getAllVLANs”.
  3. We need to differentiate the first and second run of “getAllVLANs”.  Append “1” and “2” onto the end of each def entry.
  4. Edit the VLAN name and number to whatever you want within the “createVLAN” def.
  5. Edit the list right at the bottom of the file to change what’s printed to display what we want (i.e. a main.’def’ line for each of the items above.  So, that’s “Main.getAllVLANs1”, “Main.createVLAN” and “Main.getAllVLANs2”).
  6. Run the file.

An output:

File2

File 3:

  1. Keep “getAllOrgs” and “createNewOrg” sections below the lines highlighted above and save a new file.
  2. Re-order and edit so that there is a copy of “getAllOrgs”, then the “createNewOrg” def and then another copy of “getAllOrgs”.
  3. We need to differentiate the first and second run of “getAllOrgs”.  Append “1” and “2” onto the end of each def entry
  4. Edit the Org name and tag to whatever you want within the “createNewOrg” def.
  5. Edit the list right at the bottom of the file to change what’s printed to display what we want (So, that’s “Main.getAllOrgs1”, “Main.createNewOrg” and “Main.getAllOrgs2”).
  6. Run the file.

An output:

File3

File 4:

  1. Keep “deleteVLAN” and “removeOrg” sections below the lines highlighted above and save a new file.
  2. Edit the VLAN name and then the org name to match what you created in each of the def entries.
  3. Edit the list right at the bottom of the file to change what’s printed to display what we want (“Main.deleteVLAN” and “Main.removeOrg”).
  4. Run the file.

An output:

File4

Voilà!

 

What now?

Walk through the code lines and work out what they’re doing.  Work out what classes reference what and why.  Have a play with the app.  Build on it if you feel comfortable.  Feel free to ask any questions in the comment section of this post or send me a message via “ASKSOR” which comes up when you hover over “ABOUT THE BLOGGER”.