DevOps Notes

My Linux-focused list of kernel management tasks includes:

• Following LKML (Linux Kernel Mailing List) for at least the kernel branch of my choice,
• Following NETDEV list because significant portion of my tasks revolves around network,
• Using GitWeb to port drivers and updates to the custom tree,
• Maintaining a custom kernel source tree,
• Building and packaging custom kernels to run in production.

DevOps Days 2010 had a great pannel discussion on Infrastructure as Code. The responses to one question (asked at 27:30) did surprise me. The question was "Who here goes to kernel.org grabs the sources and compiles them?" No a single panelist raised a hand. I find it shocking that all distinguished DevOps evangelists on the panel are running stock distribution kernels.

Problem

Requests to a LAMP-based Facebook application are load balanced between server nodes by an F5 BIG-IP Local Traffic Manager (LTM). The F5 BIG-IP LTM has two components: an ASIC-based fast-switching component and an AMD Opteron-based software-switching component. Layer 4 load balancing between the application server nodes can be handled either exclusively by the F5 ASIC (Performance L4 mode), or exclusively by F5 software components (Standard mode), or by a combination of F5 ASIC and F5 software components. With the Standard mode turned on, F5 BIG-IP LTM capacity utilization is at 60%. With the Performance L4 mode turned on, F5 BIG-IP LTM capacity utilization is at 20%. However, with the Performance L4 mode turned on, developers report seeing an occasional blank Facebook canvas page returned in response to legitimate application requests. Serving blank Facebook canvas responses at any significant rate is unacceptable. The issue with the blank Facebook canvas responses in Performance L4 mode has to be addressed.