monkinetic the blog

My second day at PyCon, and the start of the actual conference.

Breakfast, show floor walking. Conversation with Openstack vendor, I’m thinking we (at $DAYJOB) should commit to integrating with Oslo messaging (AMPQ) for events. The new event/messaging system is not coming any time soon.

Friday Keynote: Catherine Bracy, Code for America

> In the words of a former boss, “Technology will f*** you”.

Catherine speaks about how problems with technology intended to help (Healthcare.gov, Florida unemployment site, Arizona welfare site) end up hurting the populations they are meant to serve. We feel like government doesn’t work, when we expect to be able to access the store of the world’s knowledge from a device in our hands. It is dangerous for democracy and breeds cynicism when digital interactions with our government don’t work.

In 2015 that means that goverment must be able to do digital services well. There is no Code Red team to come save us when things go bad.

Stop Waiting for Permission - Medium post: Code for America hackathon particpant who built new tools to help people know what the parking rules are in different parts of Philly.

Detroit Water Project - CFA Fellow built a system to enable people to assist others who need help paying their water bills in Detroit. Recently graduated Y Combinator.

Local Free Web - find free internet access via text message based on bus stops.

Government can work in 21st century, but it means people who have the skills contributing to the tools and processes that need improvement.

Misc Notes:

• Code for America focuses on the improving the bureaucracy of government, not on the politics.
• Focuses on implementation and trasparenncy rather than policy.

Building Secure Systems, @lvh

The problem:

• All software has bugs
• Some of those bugs are security bugs
• All sec bugs are bad

Bad news:

• Tools don’t work
• Corporations mess up security
• The education system doesn’t educate

Tools: How do you write a test for “security”?

How do we learn software?

Read the docs, but sec docs are missing or awful. lots of misinformation. Good practice looks like bad practice.

How else do you learn: you play with it. Lather rinse repeat. but sec does not work that way. Install, play with it, get it wrong, repeat.

How else can you learn?

Install, try to break it, learn, repeat. Failure is harder, there are more failure states, issues are more subtle.

Lots of companies mess it up: no time for security testing, people make mistakes. Good rpactices look like bad practices.

What do we teach/not teach?

• Liskov Substitution: A class can be replaced by a subclass
• What don’t we teach
  • Principle of Least Authority: don’t give a system more power than it needs.

Education can be fixed

• Code review
  • disseminate experience
  • best thing after bug tracking
• adversarial approach
  • in-house (red v blue)
  • don’t stop til it doesn’t break
• automated
  • certificate checking
  • middleware tests

Meta Conversations

At this point I couldn’t keep up anymore on the slides - the presenter was flying through them and I did the best capturing the information - look for them online.

I’m asking myself what in-house apps need security-wise. How best to implement Business-Need-To-Know?

Smart Services: Microservices, Frank Stratton, Runscope

What technologies, how things are organized. Microservices are highly indenpendent codebases. Lots of moving parts though, so requires orchestration. You have to invest in infrastructure to use microservices.

Microservices == SOA + DevOps.

Example:

• Start with dashboard, wanted to build new service: identity
• Each service is on a separate host
• First step is usually:
  • http request from one service to another

But, what happens when hosts are added or change? How do you handle multiple environments?

How do you find a service?

Runscope created Smart client: a wrapper around Requests.

service://identity/<service> – this is a custom service URL that is translated at runtime by the client.

How does it find mappings for services? With *another service, Atlas*m which keeps records of mappings of hosts and services in DynamoDB.

On each service host: Sidecar, a service that runs haproxy that maps localhost service calls to the remote service. When services cahnge, haproxy is updated. Sidecar uses haproxy ACL to map services to new hosts/clusters.

Services all end up call out to localhost, which is then mapped via the HAProxy ACL to the appropriate service.

How do you make a service?

Smart service: Addons to Flask/Restful for building services. For example Runscope/healthcheck` adds healthchecks and environment data. Smart service setup returns a wrapped Flask app (like other flask addons)

Smart Config: loads configs for service – can query Atlas at runtime for base config.

current_app.name
current_app.realm  # dev, test, production, etc

current_app.redis  # redis client
current_app.db     # db client
current_app.etc    # ...

Also:

• common logging config (/var/log/runscope/etc)
• cli tool that generates a common app skeleton (sets up git repo, virtualenv, file structure)

These smart setups reduce cognitive overhead. Lots of “now I need to setup X” goes away.

Now what?

Automate everything. Puppet/etc to automate config deploy and setup. Setup monitoring, etc.

No shared databases

No shared dependencies (schemas, etc).

• Service with API
  • owned db
  • owned endpoint and load balancers
  • owned redis, etc

Makes deploying code really easy, because you are only deploying one service, not lots of other dependent code/systems.

To check out:

• Runscope/healthcheck - I already have several services this would benefit.
• HTTPbin
• RequestBin

Software Relationships! Don’t get sucked into the drama

• Deliberate Ignorance
  • Use public APIs
• Evaluate your options
  • Evaluate service dependencies
  • Evaluate library dependencies
  • Does it have API guarantees?
• Wrap difficult depencies in your own layer to protect yourself later.
  • IE, Requests wraps httplib

Protect others from your drama:

• Make sure your library/code is understandable
  • question your own assumptions
  • ignore your own internal knowledge (“oh, everybody knows X”)

At this point I have a headache from using my glasses for the computer and looking over the top of them to view the speakers. Sigh.

Your System’s “Identity”

• “utility” or “solution”
• “transparent” (users know about the things under the hood) or “encapsulated” (users don’t need domain expertise)”

[Ed. ELK stack is definitely a transparent system.]

Wrap up

About now I’m still jet-lagging, and decided that a good rest would make me right with the world for the more advanced session I want to attend tomorrow, so it’s back to the Reine Elizabeth to snooze, go over my notes, and get some dinner later.