Hi Jack, I got a call from Sandy, the secret project's PM, she says that the DEV servers are not ready yet, I really want you to understand how critical is this project for the organization, please, don't let me down... Francis, VP
Jack's boss
Hi Jack, I understand you have been working on the secret project servers setup recently, don't forget that we need to keep the documentation up-to-date yeah?
No problem! We know how to write docs.
... in the meanwhile ...
The dreaded CR
Hey Jack, we can't deploy anymore to our DEV servers. What the hell is going on?
Fixing
...things get worse...
Worst than down...
UNKNOWN STATE I
UNKNOWN STATE II
Too many people working on too many issues
Each server is managed independently and without cohesion
Chaos (r)
The "secret" project moves into QA
The GO LIVE!
Are you serious?
Jack and his team can't keep up with the work: Hundreds of servers to configure, monitor, backup, restore, fix ...
Documentation is left behind
Let's throw more people at it
...it's going to work, right?
Problems?
More people fiddling with the servers
No coordination
Recipe for disaster
What about monitoring?
Do we have it?
YES!
But...
Well...
Sorry!
No happy ending?
Where is the problem?
Communication I
Communication II
Stability vs. agility
Postpone the pain?
Fear of deploy
Because systems are fragile, each deployment is like a trip to the nearest casino
Devs and SysOps make development cycles longer to be as far as possible from the deploy date
Slower time to market, systems are down more often
Nobody is happy
Unplanned vs. failed
Unplanned vs. unique
DevOps!
Fix communication
The feedback loop
Through configuration management and deploy automation, we can deploy more often and reduce risk
The feedback loop gets shorter
Functionalities are rolled out with higher frequency
Systems are always in a known state
Changes to the system can be simulated and impacts calculated
Sharing responsibility I
Sharing responsibility II
The focus of DevOps is on automating the tasks performed during the build, QA and deployment stage
The risk of deployment errors is reduced drastically by having a strong automated testing suite, automated deployment workflow, well defined/automated rollback process
Reduce failed changes
Reduce number of unique configs
Great!
Now we've heard about DevOps!
Let's do it!
It will save us!
Let's hire a DevOps consultant!
What?
OK, we have one now...
DevOps is not a religion!
Aha!
We need an internal DevOps team!
They will work hard!
They will make DevOps happen!
Wait, another department?
What's the point?
Failed expectations
Most start with the tools
Tools are as important as...
Internal culture!
How Devs and Ops can help each other?
TALK!
TALK MORE!
SHARE!
SHARE EVERYTHING!
Hints for developers
Logging
Whenever you add new logging statement to your code, remember that the guy on the other side can actually read it
Logging level, message and frequency of logging can help or disturb
Monitoring
Embed monitoring capabilities into your code
Know monitoring channels that your operations use: JMX, SNMP, HTTP
Configuration
Structure application configuration
Backward-compatibile, good defaults, good naming
Automation
Automation over documentation
Automate everything repeatable:
build
release
deploy
test
Hints for operations
Problem solving
Get developers to solve production problems
Look at how they did it
Post-mortem analysis
Monitoring
Create dashboards! Many, but meaningful dashboards!
