Informatika | UNIX / Linux » Linux journal, 2015-03

™ Since 1994: The Original Magazine of the Linux Community A Look at the History behind systemd MARCH 2015 | ISSUE 251 | www.linuxjournalcom SYSTEM ADMINISTRATION Build Lightweight Virtual Containers PLUS Get a Fully Capable Android Tablet for $20 LJ251-March2015.indd 1 Libreboot for a Free Software Laptop WATCH: ISSUE OVERVIEW V Using Puppet’s Hiera and Encrypting Credentials 2/19/15 9:21 AM LINUX JOURNAL ARCHIVE DVD 1994–2014 A z F NOW AVAILABLE www.linuxjour nalcom/dvd LJ251-March2015.indd 2 2/19/15 9:21 AM Are you dealing with proprietary storage? you tired tieredofof dealing with proprietary storage? zStax StorCore from Silicon ® 9%2Ä4MHÆDCÄ2SNQ@FD ZFS Unified Storage From modest data storage needs to a multi-‐tiered production storage environment, zStax StorCore zStax StorCore 64 zStax StorCore 104 The zStax StorCore 64 utilizes the latest in dual-‐processor Intel® Xeon® platforms and fast SAS SSDs for caching. The zStax StorCore 64

platform is perfect for: The zStax StorCore 104 is the flagship of the zStax product line. With its highly available configurations and scalable architecture, the zStax StorCore 104 platform is ideal for: ‡VPDOOPHGLXPRIILFHILOHVHUYHUV ‡VWUHDPLQJYLGHRKRVWV ‡VPDOOGDWDDUFKLYHV ‡EDFNHQGVWRUDJHIRUYLUWXDOL]HGHQYLURQPHQWV ‡PLVVLRQFULWLFDOGDWDEDVHDSSOLFDWLRQV ‡DOZDVDYDLODEOHDFWLYHDUFKLYHV expert today:866-‐352-‐1173 866-‐352-‐1173 -‐ http://www.siliconmechanicscom/zstax TalkTalk withwith an an expert today: LJ251-March2015.indd 3 2/19/15 9:21 AM CONTENTS MARCH 2015 ISSUE 251 SYSTEM ADMINISTRATION FEATURES 58 Using Hiera with Puppet Use Hiera to encrypt sensitive data in Puppet. Scott Lackey 68 Managing Services in Linux: Past, Present and Future Learn about the history of init systems in Linux and understand how these systems evolved over time. Jonas Gorauskas 82 Infinite BusyBox with systemd Build one Linux system within another, using

the latest utilities within the systemd suite of management tools. Charles Fisher 4 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 4 2/19/15 9:21 AM COLUMNS 34 22 Dave Taylor’s Work the Shell Let’s Play Cards with Acey-Deucey, Part II 38 Kyle Rankin’s Hack and / Libreboot on an X60, Part I: the Setup 44 Shawn Powers’ The Open-Source Classroom The Teeny Tiny $20 Tablet 100 Doc Searls’ EOF Resurrecting the Armadillo 24 IN EVERY ISSUE 8 10 16 32 54 105 Current Issue.targz Letters UPFRONT Editors’ Choice New Products Advertisers Index ON THE COVER ‹(3VVRH[[OL/PZ[VY`ILOPUKZ`Z[LTKW ‹)PSK3PNO[^LPNO[=PY[HS*VU[HPULYZW ‹<ZPUN7WWL[Z/PLYHHUK,UJY`W[PUN*YLKLU[PHSZW ‹3PIYLIVV[MVYH-YLL:VM[^HYL3HW[VWW ‹7S!.L[H-SS`*HWHISL(UKYVPK;HISL[MVY W 44 LINUX JOURNAL (ISSN 1075-3583) is published monthly by Belltown Media, Inc., 2121 Sage Road, Ste 395, Houston, TX 77056 USA Subscription rate is

$2950/year Subscriptions start with the next issue WWW.LINUXJOURNALCOM / MARCH 2015 / 5 LJ251-March2015.indd 5 2/19/15 11:53 AM Executive Editor Senior Editor Associate Editor Art Director Products Editor Editor Emeritus Technical Editor Senior Columnist Security Editor Hack Editor Virtual Editor Jill Franklin jill@linuxjournal.com Doc Searls doc@linuxjournal.com Shawn Powers shawn@linuxjournal.com Garrick Antikajian garrick@linuxjournal.com James Gray newproducts@linuxjournal.com Don Marti dmarti@linuxjournal.com Michael Baxter mab@cruzio.com Reuven Lerner reuven@lerner.coil Mick Bauer mick@visi.com Kyle Rankin lj@greenfly.net Bill Childers bill.childers@linuxjournalcom Contributing Editors )BRAHIM (ADDAD s 2OBERT ,OVE s :ACK "ROWN s $AVE 0HILLIPS s -ARCO &IORETTI s ,UDOVIC -ARCOTTE 0AUL "ARRY s 0AUL -C+ENNEY s $AVE 4AYLOR s $IRK %LMENDORF s *USTIN 2YAN s !DAM -ONSEN President Carlie Fairchild publisher@linuxjournal.com Publisher Mark Irgang

mark@linuxjournal.com Associate Publisher John Grogan john@linuxjournal.com Director of Digital Experience Accountant Katherine Druckman webmistress@linuxjournal.com Candy Beauchamp acct@linuxjournal.com Linux Journal is published by, and is a registered trade name of, Belltown Media, Inc. PO Box 980985, Houston, TX 77098 USA Editorial Advisory Panel Nick Baronian Kalyana Krishna Chadalavada "RIAN #ONNER s +EIR $AVIS -ICHAEL %AGER s 6ICTOR REGORIO $AVID ! ,ANE s 3TEVE -ARQUEZ $AVE -C!LLISTER s 4HOMAS 1UINLAN Chris D. Stark Advertising % -!),: ads@linuxjournal.com URL: www.linuxjournalcom/advertising 0(/.%     EXT  Subscriptions % -!),: subs@linuxjournal.com URL: www.linuxjournalcom/subscribe MAIL: PO Box 980985, Houston, TX 77098 USA LINUX is a registered trademark of Linus Torvalds. LJ251-March2015.indd 6 2/19/15 9:21 AM LJ251-March2015.indd 7 2/19/15 9:21 AM Current Issue.targz Putting Out Fires and Designing FireProof Buildings SHAWN POWERS S

ystem administration is a very general term. It’s our job to fix problems, repair systems and remind people to try power cycling their troubled desktops. We are also responsible for creating systems that don’t develop problems, need fewer repairs and run without being power cycled. In an ideal world, system administrators would work themselves out of a job in short order. Thankfully (or unfortunately?), that’s not how it goes. We always have problems to fix, and there’s always a better way to do what we’re doing. Thus, system administration is a vibrant V VIDEO: Shawn Powers runs through the latest issue. and ever-changing field. This month, we learn how to be better at our jobs, even if the measure of “success” is constantly fluctuating. Dave Taylor starts off this issue with a continuation of his script-based card game. Designing games with Dave is a great way to become better shell scripters, and so in a very real sense, we can justify playing games at work. Kyle

Rankin follows Dave with a nerdier sort of game: trying to replace the proprietary BIOS on a ThinkPad with Libreboot. Coreboot is an open-source BIOS replacement, and Libreboot goes a step further by stripping out all the proprietary code. If you think having a free BIOS with built-in GRUB sounds interesting, you’ll want to check out Kyle’s column this month. 8 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 8 2/19/15 9:21 AM CURRENT ISSUE.TARGZ My personal contribution to the System Administration issue is something I find to be more useful than I ever expected. Android tablets are convenient for things like Wi-Fi sniffing, but they are often unwieldy to carry around. My solution is to convert a cheap pre-paid cell phone into a tiny, pocket-size tablet. If you already have an Android phone, it might be redundant, but for me, a $20 tablet was too hard to pass up. In my column, I give you all the details. Puppet is an incredible tool for managing the system

configurations of multiple nodes. Scott Lackey describes a great tool we can use to store sitespecific data more efficiently (and securely). Hiera is a key/value lookup tool that integrates directly with Puppet and makes a great tool even better. If you want to have a clear separation between your sensitive data and the Puppet system that uses it, or if you want to save time by reusing common data, Hiera is a tool any Puppet admin will want to check out. Jonas Gorauskas gives us a history of systemd. Whether you love the new initialization system, or think it’s a terrible implementation of a horrible idea, systemd is here to stayat least for a while. If you’ve ever been curious how we got FROM SIMPLE INIT SCRIPTS TO 3YS6 AND beyond, you’ll want to read Jonas’ article. Once you understand systemd, Charles Fisher follows up with a great tutorial on using the new init system to create powerful and lightweight virtual containers utilizing systemd FOR INITIALIZATION &OR

STUBBORN 3YS6 lovers like myself, it’s great to read some information on the advantages systemd might offer. Doc Searls closes out our issue with a new look at the 15-year-old Cluetrain Manifesto. If you’re a fan of the Locke, Levine, Weinberger and Searls project, you’ll want to read what’s happening with New Clues today. If it weren’t for the modern technological world we live in, system administration wouldn’t even exist! Thankfully (or again, unfortunately?), our world is getting more and more technological every day. The need for system administrators and their tools are more in demand than ever before, and this issue of Linux Journal was written to educate, inform and even entertain those of us in the digital trenches. We hope you enjoy this issue as much as we enjoyed putting it together! Q Shawn Powers is the Associate Editor for Linux Journal . He’s also the Gadget Guy for LinuxJournal.com, and he has an interesting collection of vintage Garfield coffee mugs.

Don’t let his silly hairdo fool you, he’s a pretty ordinary guy and can be reached via e-mail at shawn@linuxjournal.com Or, swing by the #linuxjournal IRC channel on Freenode.net WWW.LINUXJOURNALCOM / MARCH 2015 / 9 LJ251-March2015.indd 9 2/19/15 9:21 AM letters I’m glad you’re back in the fold, welcome home!Shawn Powers Vagrant Simplified 2EGARDING 3HAWN 0OWERS h6AGRANT Simplified” in the January 2015 ISSUE GREAT ARTICLE ) TRIED 6AGRANT a few months back, and I couldn’t get the light bulb to turn on. Thus, I put it aside. Shawn’s article supplied the understanding I was missing. Many thanks Tim Parks Digital Format An interesting thing happened. I dropped reading LJ a while back due to hating to stare into a monitor. But last week I finally purchased an Amazon tablet and re-subscribed to LJ because of the LJ app. It’s now easy TO READ ON A NICE SCREEN %VEN THOUGH I still enjoy printed magazines, I do respect the environment and agree that chopping down green

for this is not good. So good choice on an environmentally-friendly mag. Peter K. Thanks Peter! Paper magazines have a dear place in my heart as well, but I can’t deny the digital format has some advantages too. That’s exactly what I was hoping for! I’m glad it worked, and I’m glad Vagrant is demystified for a few more people. Thank you for the kind words.Shawn Powers Suggestion for Dave Taylor ! WHILE AGO ) WROTE A # PROGRAM for downloading stock and option information from Yahoo in Windows. I remember it took a lot of code to parse the information, most particularly the option information. Since then, I have graduated to Linux, and I am currently running on Xubuntu. ! FRIEND PIQUED MY INTEREST IN OPTION trading that caused me to revisit coding a Linux version of option tracking. I have 10 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 10 2/19/15 9:21 AM [ LETTERS not yet gotten around to coding any GUI STUFF AND ) WANTED SOMETHING QUICKLY SO ) JUST WROTE A

FEW # PROGRAMS USING Geany to run on the terminal. 0.00 I really didn’t want to write hundreds of lines of code to parse the data and looked around for some XML parsers. None looked easy enough for me to use, but then I looked at the source page, which I downloaded, and tried grep , which led me to develop the following few, or one, line(s) of code that I thought you might be interested as a source for some future articles: Of course, with a slight change to: wget -O /tmp/ option.html ´http://finance.yahoocom/q/op?s=SPY&date=1429228800 grep option entry|:volume /tmp/ option.html | sed -n ´s/ //;s/[^>]*//;s/>//;s/</div>//;p | sed ´s/</strong>//;s/[^>]*>//;s/</a>//;s/%// > /tmp/ option.txt ] 10 21 65.14 wget -q -O /tmp/ option.html ´http://finance.yahoocom/q/op?s=SPY&date=1429228800 ´&& sleep 1 && grep ´option entry|:volume /tmp/ option.html | sed -n ´s/ //;s/[^>]*//;s/>//;s/</div>//;p | sed

´s/</strong>//;s/[^>]*>//;s/</a>//;s/%// ´| grep -A8 SPY141226P00230000 I could say that we can use “one line” of code to parse the Yahoo finance page for the option information! 4HIS IS # IN THAT ) USE SYSTEM TO execute. Maybe a popen function might be better as an alternative, but I didn’t think of it at the time. grep -A8 SPY141226P00230000 option.txt These three lines extracted the following information: SPY141226P00230000 24.27 22.87 24.09 0.00 In summary, I thought this was pretty cool, and you may have already done something similar, but as I said, I thought it might give you some ideas for future articles. I enjoy and find your articles educational, which are usually one of the first I read after the titles that catch my eye. Roger WWW.LINUXJOURNALCOM / MARCH 2015 / 11 LJ251-March2015.indd 11 2/19/15 9:21 AM [ LETTERS ] Dave Taylor replies: Thanks for your note and code snippet, Roger. It is rather amazing what you can do with sed,

although when it gets that complex, you might consider having the script in a separate file and using the -f FILE option to sed to retain your sanity as you debug it. The problem with all of these crude HTML parsers, of course, is that if they make the slightest tweak on the page, your code’s broken. I know; it happens to me all the time. Response to Letter in the January 2015 Issue Regarding zbackup Regarding Chris W ills’ letter in the January 2015 issue [this letter is from David Barton, author of the article “Ideal Backups with ZBACKUPv IN THE .OVEMBER  issue]: currently I use rsnapshot to back up the zbackup stores with hourly, daily, weekly rotations. Because the zbackup store changes very slightly each time, it is very space-effective. Due to the IO load caused by large numbers of files, very large numbers of servers may want to look at options that don’t REQUIRE LINKING ALL THE FILES SUCH AS rotating thin provisioned snapshots, "TRFS:&3 SNAPSHOTS OR

ROTATING ONTO removable storage media like tape. I don’t think the snapshots need to be replicated, since it is a guard against malicious file corruption for example, an administrator inserting random bits into the files. Also, for readers who are interested in using zbackup to back up very large directory structures, there is a pre-release of software on https://github.com/ davidbartonau/zbackup-tar that backs up directories about 10x faster on a non-SSD. David Barton Kyle Rankin’s Dr Hjkl on the Command Line Regarding Kyle Rankin’s article “Dr Hjkl on the Command Line” in the $ECEMBER  ISSUE OF LJ: it seems that Mr Rankin wants to use vi keystrokes to manipulate the shell command line, so why is he explaining %MACS MODE KEYSTROKES )N THE SHELL all he needs to type is set -o vi and use vi mode from then on. Hit %SC TO ENTER COMMAND MODE THEN hop to the previous word with b, next word with w, change the current word WITH CW AND SO ON %VEN hHJKLv ARE active, for moving

the cursor to the previous/next letter/shell command. The mechanism is called GNU READLINE IT SUPPORTS BOTH %MACS (default) and vi mode (put set editing-mode vi in ~/.inputrc), 12 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 12 2/19/15 9:21 AM [ and most command-line tools like the SHELL OR THE -Y31, CLIENT OR THE $" debugger will behave accordingly because they’re using the library. Mike Dr Hjkl on the Command Line, II )N THE $ECEMBER  ISSUE SERIOUSLY Kyle: set -o vi . Xaveer Dr Hjkl on the Command Line, III )N +YLE 2ANKINS $ECEMBER  column, he describes being comfortable LJ251-March2015.indd 13 LETTERS ] with the vi command set, as I am too. He then goes on to describe bash command-line editing capabilities, using lots of Ctrl and Alt keys. If you’ve ever been sucked into editor wars, I’m sure it occurred to you that those key SEQUENCES SEEM AWFULLY %MACS LIKE And, in fact, that’s exactly what they are. Bash starts out with its

command-line EDITING IN %MACS MODE (OWEVER BASH also has a perfectly functional vi mode that may seem more familiar to you. Just do set -o vi to turn on vi mode. 2/19/15 9:21 AM [ LETTERS ] Now, having said that, I have to admit that I leave my bash sessions IN %MACS MODE ALMOST EXCLUSIVELY The vi mode, like the vi editor, is modal, and that modality is somewhat non-intuitive in command-line editing. However, for a vi fan, it’s certainly worth exploring. Tim Roberts Kyle Rankin replies: I remember when I first got really interested in vi that I changed the command line to vi mode. I realized pretty quickly though that I didn’t like having modes on the command line and switched it back. In general, I try to keep my environments set to their defaults, so you won’t find me with custom bashrc files that set a lot of aliases or anything like that. It’s just too much of a pain to ship custom settings like that throughout all my home and work systems, so instead, I try to

make the most with the defaults I get. Digital Format Happy New Year to you and your team. I had stop subscribing to this magazine some time ago and came back because of the digital format. Why? Because I am a seaman who is away for six months at a time. W ithout the digital format, it’s hard to keep reading such a fine magazine. Seriously, I do not understand 60% of what is written, but if one keeps reading, surely one’s knowledge will gradually improve. Keep up the good work KokYY Awesome! Yes, please keep reading. Then after a couple months go back and see if any of the older stuff makes sense. (Don’t worry if it doesn’t all make sense, however; sometimes the articles make my head spin too!)Shawn Powers They Said It I just wanted to put in a good word for the They Said It column. 4HE QUOTES ARE NOT ALWAYS memorable (although they often are), but they always put me in a good frame of mind for enjoying the rest of the issue. Steven Janke Thank you Steven. I enjoy looking for

good quotes every month. The hardest part is making sure I don’t repeat any (unless they’re really good ones!)Shawn Powers Kyle Rankin’s EC2 Security Groups In the “Secure Server Deployments 14 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 14 2/19/15 9:21 AM in Hostile Territory” article in the January 2015 issue, Mr Rankin says: “. it’s important to know that Security Groups are assigned only when an instance is createdyou can’t add or remove Security Groups from an instance after you create it.” )N A 60# WHICH HAS BEEN %#S DEFAULT FOR A good while now, this is not true. You easily can change the Security Group(s) associated with an instance. Gx Kyle Rankin replies: Thanks for the e-mail. It could be that I’m just showing my age in how long I’ve been working with EC2, since none of my accounts default into VPCs, and I’m still in the wild west of “EC2 Classic”. That said, I do think having to think in terms of the limitations of the

classic EC2 Security Group model helps build more robust security since you can take less for granted. WRITE LJ A LETTER We love hearing from our readers. Please send us your comments and feedback via http://www.linuxjournalcom/contact PHOTO OF THE MONTH Remember, send your Linux-related photos to ljeditor@linuxjournal.com! At Your Service SUBSCRIPTIONS: Linux Journal is available in a variety of digital formats, including PDF, .epub, mobi and an on-line digital edition, as well as apps for iOS and Android devices. Renewing your subscription, changing your e-mail address for issue delivery, paying your invoice, viewing your account details or other subscription inquiries can be done instantly on-line: http://www.linuxjournalcom/subs E-mail us at subs@linuxjournal.com or reach us via postal mail at Linux Journal, PO Box 980985, Houston, TX 77098 USA. Please remember to include your complete name and address when contacting us. ACCESSING THE DIGITAL ARCHIVE: Your monthly download

notifications will have links to the various formats and to the digital archive. To access the digital archive at any time, log in at http://www.linuxjournalcom/digital LETTERS TO THE EDITOR: We welcome your letters and encourage you to submit them at http://www.linuxjournalcom/contact or mail them to Linux Journal, PO Box 980985, Houston, TX 77098 USA. Letters may be edited for space and clarity. WRITING FOR US: We always are looking for contributed articles, tutorials and real-world stories for the magazine. An author’s guide, a list of topics and due dates can be found on-line: http://www.linuxjournalcom/author FREE e-NEWSLETTERS: Linux Journal editors publish newsletters on both a weekly and monthly basis. Receive late-breaking news, technical tips and tricks, an inside look at upcoming issues and links to in-depth stories featured on http://www.linuxjournalcom Subscribe for free today: http://www.linuxjournalcom/ enewsletters. ADVERTISING: Linux Journal is a great resource for

readers and advertisers alike. Request a media kit, view our current editorial calendar and advertising due dates, or learn more about other advertising and marketing opportunities by visiting us on-line: http://ww.linuxjournalcom/ advertising. Contact us directly for further information: ads@linuxjournal.com or +1 713-344-1956 ext. 2 WWW.LINUXJOURNALCOM / MARCH 2015 / 15 LJ251-March2015.indd 15 2/19/15 9:21 AM UPFRONT NEWS + FUN diff -u WHAT’S NEW IN KERNEL DEVELOPMENT Nicolas Dichtel and Thierry Herbelot pointed out that the directories in the /proc filesystem used a linked list to identify their files. But, this would be slow when /proc directories started having lots of files, which, for example, might happen when the system needed lots of network sockets. Nicolas and Thierry posted a patch to change the /proc implementation to use multiple linked lists instead of just one. %ACH SUBDIRECTORY WOULD HAVE ITS own linked list, keyed to a hash of the directory’s name.

According to their benchmarks, the patch shaved 1/5 of the time needed to chur n through all the entries of a given subdirectory. Stephen Hemminger liked the speedup, but suggested that there already were implementations, like the hlist macro, that might simplify their hash table code. Eric W. Biederman also liked the speedup and kicked himself for overlooking the /proc issue when doing other scalability work. But, he felt that the whole linked list concept was not the right APPROACH %SPECIALLY HE FELT THAT /proc/net/dev/snmp6 was the real target of Nicolas and Thierry’s patch, and if no one actually needed the files in that directory EXCEPT PEOPLE REQUIRING EXTREME backward compatibility), it would be even more efficient to do away with them completely. This, however, already had come up in an earlier thread, when David S. Miller had said that “It potentially breaks tools, it’s a non-starter, sorry.” So, reworking the user interface would not be allowed, which left the

linked list speedup that Nicolas and Thierry proposed. But, Nicolas said he’d look into an rbtree implementation instead of a plain linked list, because rbtrees would potentially scale better. Minchan Kim noticed that putting memory pressure on qemu-kvm UNDER ,INUX  WOULD cause a ker nel stack overflow and crash the system. He dug into the 16 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 16 2/19/15 9:21 AM [ code and tried to reduce his own stack usage, but he wasn’t able to cut back enough to prevent the crash. And in any case, he said, trying to reduce everyone’s stack usage was not very scalable. He proposed expanding the ker nel stack from 8K to 16K, although he acknowledged that there possibly were good reasons not to do this that he wasn’t aware of. Dave Chinner remarked that “8k stacks were never large enough to fit the Linux IO architecture UPFRONT ] ON X  BUT NOBODY OUTSIDE filesystem and IO developers has been willing to accept that

argument as valid, despite regular stack overruns and filesystems having to add workaround after workaround to prevent stack overruns.” He added, “We’re basically at the point where we have to push EVERY 8&3 OPERATION THAT REQUIRES block allocation off to another thread to get enough stack space for normal operation”, and said LINUX JOURNAL now available for the iPad and iPhone at the App Store. linuxjournal.com/ios For more information about advertising opportunities within Linux Journal iPhone, iPad and Android apps, contact John Grogan at +1-713-344-1956 x2 or ads@linuxjournal.com LJ251-March2015.indd 17 2/19/15 9:21 AM [ UPFRONT ] “XFS has always been the stack usage canary and this issue is BASICALLY A REPEAT OF THE K STACK on i386 ker nel debacle.” Borislav Petkov pointed out that if they increased the ker nel stack from 8K to 16K, there undoubtedly would come a time when 16K wouldn’t be enough either. He wondered if there ever would be a limit, or

if the ker nel stack ultimately would grow to one megabyte and beyond. Steven Rostedt said, “If [Minchan’s patch] goes in, it should be a config option, or perhaps selected by those filesystems that need it. I hate to have 16K stacks on a box that doesn’t have that much memory, but also just uses ext2.” Meanwhile, H. Peter Anvin said, “8K additional per thread is a huge hit. XFS has indeed always been a canary, or trouble spot, I suspect because it originally came from another ker nel where this was not an optimization target.” At around this point, Linus Torvalds remarked that something like Minchan’s fix probably would be necessary at some point, although the development cycle was already at -rc7, making it too late for that particular ker nel version. Linus also pointed out that there was plenty of room to reduce stack usage in the stack trace Minchan had posted in his original e-mail. Linus remarked, “From a QUICK GLANCE AT THE FRAME USAGE some of it seems to be

gcc being rather bad at stack allocation, but lots of it is just nasty spilling around the disgusting call-sites with tons or arguments. A lot of the stack slots are marked as ’%sfp’ (which is gcc-ese for ’spill frame pointer’, afaik).” There was a technical discussion about various ways to reduce stack usage in general (and some further consideration of ways in which GCC might be somewhat to blame), but with Linus willing to accept a patch to implement a larger stack, it seems like something along the lines of Minchan’s patch will soon be part of the ker nel. At one point, Linus summed up his position on the issue, saying, “Minchan’s call trace and this thread has actually convinced me that yes, we really do need to make X  HAVE A K" STACK ;= The 8kB stack has been somewhat restrictive and painful for a while, and I’m ok with admitting that it is just getting too damn painful.” ZACK BROWN 18 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd

18 2/19/15 9:21 AM March 16–19, 2015 | Boston, MA From strategies to essential technologiesbuild a solid foundation in software architecture The O’Reilly Software Architecture Conference is a new event designed to provide the in-depth professional training that software architects and people working on software architecture need to support the success of their businesses. ■ ■ ■ ■ ■ ■ ■ Reactive and its variants Microservices Continuous Delivery Integration Architecture Devops Scaling Big Data ■ ■ ■ Continuous Deployment Architecture Fundamentals Business Skills Save 20% softwarearchitecturecon.com @oreillysacon LJ251-March2015.indd 19 on your ticket Use code LINUXJ 2/19/15 9:21 AM [ UPFRONT ] Android Candy: Bluetooth Auto Connect I love my latest Android device (see this issue’s Open-Source Classroom column for details), but for some reason, it won’t automatically connect to my Bluetooth headset. When I turn on my headset, I want

it to connect to my Android device so I can start using it right away. In order to make it connect, I have to go into the settings app, then Bluetooth, and then tap the device to connect. Thankfully, there’s an application that makes life a lot easier. Bluetooth Auto Connect is a program that runs in the background. It doesn’t constantly poll for newly turned on Bluetooth devices, because that would waste battery power. It has several other ways to initiate the connection though. My favorite is the “connect when powered on” option. Because I always have to turn the phone on in order to start my audiobook (or music), it’s not an inconvenience to turn the screen on in order to connect Bluetooth. As soon as the power button is pressed, it connects to my headset, and by the time I open the media player application, it’s ready to rock! Sometimes it’s the simplest applications that are the most useful. Bluetooth Auto Connect is one of those. Check it out in the Google Play

Store today: https://play.googlecom/store/apps/ details?id=org.myklosbtautoconnect SHAWN POWERS They Said It Do something every day that you don’t want to do; this is the golden rule for acquiring the habit of doing your duty without pain. Mark Twain It’s okay if you mess up. You should give yourself a break. Billy Joel Let me tell you the secret that has led me to my goal. My strength lies solely in my tenacity. Louis Pasteur If you limit your choices only to what seems possible or reasonable, you disconnect yourself from what you truly want, and all that is left is a compromise. Robert Fritz The highest result of education is tolerance. Helen Keller 20 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 20 2/19/15 9:21 AM LJ251-March2015.indd 21 2/19/15 9:21 AM [ UPFRONT ] Non-Linux FOSS: MenuMeters It sounds like a “back in my day” story, but I really do miss the days when LAPTOPS HAD ,%$ ACTIVITY LIGHTS FOR HARD drives and Wi-Fi. Sure, some still have

Menu Bar (screenshot from http://ragingmenace.com) Customizing MenuMeters 22 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 22 2/19/15 9:21 AM [ them, but for the most part, the latest trend is to have no way of knowing if your application is pegging the CPU at 100%, or if it just locked up. The hardware on Apple-branded LAPTOPS IS AMAZING %VEN IF YOU HATE the operating system, the solid aluminum cases are just awesome. Like most other brands of laptops, however, they lack any activity lights. A perfect fix for OS X is the open-source UPFRONT ] MenuMeters application. It puts all sorts of monitoring ability right in your menu bar. MenuMeters supports CPU activity, network activity and even memory usage. With a wide range of display options, you can customize MenuMeters to be as informative or subtle as you like. MenuMeters is licensed under the GPL and is available to download at http://www.ragingmenacecom SHAWN POWERS Tighten Up SSH SSH is a Swiss Army knife and

Hogwart’s magic wand all rolled into one simple command-line tool. As often as we use it, we sometimes forget that even our encrypted friend can be secured more than it is by default. For a full list of options to turn on and off, simply type man sshd config to read the man page for the configuration file. As an example, one of the first things I do is disable root login via SSH. If you open /etc/ssh/sshd config as root, search for a line mentioning PermitRootLogin and change it to no . If you can’t find a line with that option, just add it to the end. It will end up looking like: PermitRootLogin no Plenty of other security options are available as well. Disabling the old SSH version 1 protocol is as simple as changing (or adding): Protocol 2, 1 Change it to: Protocol 2 Then only the far more secure version  PROTOCOL WILL BE ABLE TO CONNECT %VERY server situation has different security needs. Reading through the man page might reveal some options you never even considered

before. (Note that the sshd dæmon will need to be restarted for the changes to be applied. Or, if in doubt, just reboot the computer.) SHAWN POWERS WWW.LINUXJOURNALCOM / MARCH 2015 / 23 LJ251-March2015.indd 23 2/19/15 9:21 AM [ UPFRONT ] Solving ODEs on Linux Many problems in science and engineering are modeled through ORDINARY DIFFERENTIAL EQUATIONS /$%S http://en.wikipediaorg/wiki/ Ordinary differential equation). !N /$% IS AN EQUATION THAT CONTAINS A function of one independent variable and its derivatives. This means that practically any system that changes over time can be modeled with an /$% FROM CELESTIAL MECHANICS TO chemistry reaction rates to ecology and population modeling. "ECAUSE OF THIS UBIQUITY MANY TOOLS have been developed through the years TO HELP SOLVE AND ANALYZE /$%S )N this article, I take a look at one of the tools available on Linux: Model Builder (http://model-builder.sourceforgenet) The project is hosted on SourceForge, so you always can build

it from source, but most distributions should have a package available. On Debianbased distros, you can install it with the command: sudo apt-get install model-builder It also installs several Python modules to support the tasks it can handle. If you do decide to build from source, you will need to handle these dependencies yourself. Included with the source is a directory of examples. You can use them as a starting point and to gain some ideas of what you can do with Model Builder. Documentation is a bit sparse, so you may need to get your hands a little dirty to take the most advantage of what is possible with Model Builder. To start Model Builder, you either can click on its menu item in your desktop environment or run the command PyMB from a terminal window. When the main window pops up, you are presented with a template where you can define the problem you are analyzing (Figure 1). The main pane, titled $IFFERENTIAL %QUATIONS IS WHERE you can define the set of ordinary

DIFFERENTIAL EQUATIONS THAT YOU ARE trying to solve. The general form of THESE EQUATIONS IS DYDT  FY T  If your system depends on different levels of differentiating the dependent variable, you always CAN REWRITE IT AS A SYSTEM OF /$%S When you give Model Builder your system, you need to write out only the right-hand side of the ABOVE EQUATION 4HIS EQUATION CAN 24 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 24 2/19/15 9:21 AM [ UPFRONT ] Figure 1. When Model Builder starts, you can set several parameters and the equations you want to analyze. contain essentially any function or expression that NumPy understands, since Model Builder uses Python to do the heavy lifting. Because Model Builder is designed TO HANDLE SYSTEMS OF EQUATIONS you need to define the y portion as elements of a list. So the y variable FOR THE FIRST EQUATION IS LABELED AS y[0]; the y variable for the second EQUATION IS LABELED Y;= AND SO ON These are called the state variables. The pane

to the right of the EQUATION WINDOW IS WHERE YOU CAN place any parameters that you need, one per line. They can be used in THE EQUATION WINDOW WHERE THEY are labeled as p[0], p[1] and so on. If you want to use time in either the PARAMETERS OR EQUATIONS THAT YOU have defined, you just need to use the t variable. Because Python is used in the back end, you even can use lambda WWW.LINUXJOURNALCOM / MARCH 2015 / 25 LJ251-March2015.indd 25 2/19/15 9:21 AM [ UPFRONT ] functions to define more complex structures. You may want to take a look at the documentation available on the NumPy site to see what options are available (http://www.numpyorg) Below these two panes is where you define the rest of the options for your problem. In the Initial values box, you can enter the initial values for each state variable at THE TIME T 4HEY NEED TO BE separated with a space and put in THE ORDER OF THE EQUATIONS GIVEN IN THE EQUATION PANE Below the Initial values, you can enter the start time,

the end time and the time step to use in the solution. The critical time steps box is usually left empty, so let’s Figure 2. Once you finish defining the problem and run the integration, a result window pops up with a graph of the integration. 26 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 26 2/19/15 9:21 AM [ UPFRONT ] Figure 3. You always can get a typeset display of your equations to verify what they should look like. leave it alone here. The first step box is the size of the first step. Usually, you should leave this as 0 to allow for automatic determination. The minimum and maximum step size boxes set these variables that are used in the variable step size algorithm. Typically, you should leave these as 0 as well to allow for automatic determination. The full output check box will print out more useful information about the integration in the results spreadsheet. Once everything is entered, all you need to do is click the Start icon, and the integration

will be calculated. If this is a system that you will want to work with over time, you can click on the menu WWW.LINUXJOURNALCOM / MARCH 2015 / 27 LJ251-March2015.indd 27 2/19/15 9:21 AM [ UPFRONT ] Figure 4. You can pull up all of the results of your integration and do further analysis item FileASave to save the model to a file. This file format is an XML file, so you could edit it with a text editor if you want. When you are ready to do more work with it, you can load it by clicking on FileAOpen. Once the calculations are done, which may be fast for simple problems, a results window will pop up (Figure 2). matplotlib handles this graph window, so you can manipulate it just like any other matplotlib window. This includes panning, zooming or changing the plot window. You also can save the resulting plot as an image file in one of several different formats. Going back to the main window, let’s look at some other available tools. Clicking on the Show EQUATIONS ICON POPS UP A

WINDOW 28 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 28 2/19/15 9:21 AM [ UPFRONT ] Figure 5. You can generate a power spectrum of any column of your results WHERE YOU CAN SEE THE EQUATIONS typeset (Figure 3). Beside this icon is the Results icon. Clicking on that pops up a spreadsheet of all of the results from your integration &IGURE   4HE COLUMNS OF DATA include the time, the value of y[0] and the step sizes, among other things. You can select a couple columns by holding down the Ctrl key and clicking on the column headers. Then, click on the plot button to plot them in a new window. You can get a power spectrum for any one column by selecting one of interest and clicking on the Spectrum icon. This pops up two new windows, the first a power spectrum of the column (Figure 5) and the second a spectrogram of the column (Figure 6). The last tool available is a wavelet WWW.LINUXJOURNALCOM / MARCH 2015 / 29 LJ251-March2015.indd 29 2/19/15 9:21 AM [

UPFRONT ] Figure 6. You also can generate a spectrogram of your results transform. When you select a column, you can apply a continuous wavelet transform to the data. When you are done with Model Builder, you can save this data into A COMMA SEPARATED VALUES #36 FILE from the spreadsheet window. Then, you can import it into other tools, like R, to do even further analysis. Now that you have seen the options available in Model Builder, hopefully you will consider it WHEN LOOKING AT /$% PROBLEMS It provides a pretty simple interface to the tools available in Python to SOLVE /$%S NUMERICALLY !LTHOUGH other more powerful tools are available, Model Builder fits into THE NICHE OF EXPERIMENTING QUICKLY WITH DIFFERENT EQUATIONS AND PLAYING with ideas. JOEY BERNARD 30 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 30 2/19/15 9:21 AM 15HPCLinux-LinuxJnlMarchad.qxp Layout 1 2/10/15 3:34 PM Page 1 12th Annual C F Re onf ree w ua gi ere w lif str nc w. G fla o ied ati e gg O

E on m nli nd gm ne U t.c - sers Roosevelt Hotel, NYC . om Madison Ave and 45th St, next to Grand Central Station /li nu x 2015 HPC FOR WALL STREET – fo rQ CLOUD TECHNOLOGY April 6, 2015 (Monday) Plan to Attend: 2015 HPC for Wall Street will deliver top-notch content and connections. Low-cost conference at $295. save $100 Full program, including lunch. Dave Weber Ken Barnes Financial Services SVP Corp Dev, Options Free Conference Registration for quali- Global Segment Leader, Lenovo Information Technology fied end users. Register online as end user. Bernard S Donefer Associate Director, Baruch College Mike Blalock Global Sales Director, Intel Cloud Technology,Big Data, Low Latency, Networks, Data Centers, APIs, Scalability, cost savings for the global financial markets. Leading Wall Street IT directors and vendor technology experts will speak on the program. Paul Jameson Managing Director, Global Fin Services, Cisco Systems Dave Malik Senior Director, Advanced Services,

Cisco Systems Dino Vitale Dir, Morgan Stanley Quality Assurance & Production Mgmt Harvey Stein Head of Credit Risk Modeling, Bloomberg Fadi Gebara Sr Manager, IBM Research Terry Keene CEO, iSys Rob Krugman VP Digital Strategy, Broadridge Fin Sols Lee Fisher VP Marketing, Redline Trading Solutions Jeremy Eder Perf Engineering, Red Hat Matt Smith Sol Architect, Red Hat David B. Weiss Sr Analyst, Aite Rick Aiere Architect Specialty, AIG Jeffrey Scheel Senior Technical Staff, IBM Linux Tech Center Ed Turkel Mgr WW HPC Mkting, Hewlett-Packard Charles Milo Enterprise Technical Specialist, Intel Davor Frank Sr Solutions Architect, Solarflare Phil Albinus Editor, Traders Magazine, SourceMedia Speakers will cover 2015 Cloud, HPC and the latest programs to increase speed, put-through, and reduce costs. Full conference program includes industry luncheon, general sessions, drill down sessions, exhibits, post show receptions. Don’t have time for the full Conference? Attend the

free Show. Register in advance at: wwwflaggmgmtcom/linux 2015 Sponsors ™ Shagun Bali Analyst, TABB Group Show Hours: Mon, April 6 Conference Hours: Mon, April 6 8:00 - 4:00 8:30 - 4:50 Visit: www.flaggmgmtcom/linux LJ251-March2015.indd 31 Show & Conference: Flagg Management Inc 353 Lexington Avenue, New York 10016 (212) 286 0333 fax: (212) 286 0086 flaggmgmt@msn.com 2/19/15 9:22 AM [ EDITORS CHOICE ] ™ NmapNot Just for Evil! If SSH is the Swiss Army knife of the system administration world, Nmap is a box of dynamite. It’s really easy to misuse dynamite and blow your foot off, but it’s also a very powerful tool that can do jobs that are impossible without it. When most people think of Nmap, they think of scanning servers, looking for open ports to attack. Through the years, however, that same ability is incredibly useful when you’re in charge of the server or computer IN QUESTION 7HETHER YOURE TRYING to figure out what kind of server is using a specific

IP address in your network or trying to lock down a new NAS device, scanning networks is incredibly useful. Figure 1 shows a network scan OF MY 1.!0 !3 4HE ONLY THING ) use the unit for is NFS and SMB file sharing, but as you can tell, it has a ton of ports wide open. W ithout Nmap, it would be difficult to figure out what the machine was running. Another incredibly useful way to use Nmap is to scan a network. EDITORS’ CHOICE ★ You don’t even have to have root access for that, and it’s as simple as specifying the network block you want to scan. For example, typing: nmap 192.16810/24 WILL SCAN THE ENTIRE RANGE OF  possible IP addresses on my local network and let me know which are pingable, along with which ports are open. If you’ve just plugged in a new piece of hardware, but don’t know what IP address it grabbed via DHCP, Nmap is priceless. For example, the above command revealed this on my network: Nmap scan report for ´TIVO-8480001903CCDDB.brainofshawncom

(1921681220) Host is up (0.0083s latency) Not shown: 995 filtered ports PORT STATE SERVICE 80/tcp open http 443/tcp open https 2190/tcp open tivoconnect 2191/tcp open tvbus 9080/tcp closed glrpc 32 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 32 2/19/15 9:22 AM Figure 1. Network Scan This not only tells me the address of my new T ivo unit, but it also shows me what ports it has open. Thanks to its reliability, usability and borderline black hat abilities, .MAP GETS THIS MONTHS %DITORS Choice award. It’s not a new program, but if you’re a Linux user, you should be using it! SHAWN POWERS WWW.LINUXJOURNALCOM / MARCH 2015 / 33 LJ251-March2015.indd 33 2/19/15 9:22 AM COLUMNS WORK THE SHELL Let’s Play Cards with Acey-Deucey, Part II DAVE TAYLOR Dave adds the necessary code to turn a demo into a playable game, complete with some rule variants. In my last article, I started developing a simple card game called Acey-Deucey, in which you deal two

cards face up, then bet on whether the next card is going to be between those two in rank value. In other words, if a 5 of diamonds and a jack of spades were flipped up, the bet would be whether the next card was going to be between a 6 and a 10. I also dug into the math too, if you missed it, because this is a great game for understanding odds and probability. Remember, any given card has a 1 in 52 chance of appearing, and because two cards already have been exposed, that means any given card actually has 1:50 odds. For the example above, there are four 6s, four 7s, 8s, 9s and 10s, MEANING THAT THERE ARE       OR A  CHANCE THAT the next card flipped up will indeed be between the two exposed cards. Make that 5 of diamonds an ace of diamonds, and the odds get crazy good: 80%. I’d take those odds! The math will factor into the script because you actually can have the game suggest what to do based on the odds. The greater the spread, the better the odds easy enough. I

ended my last article with the game being able to shuffle and deal three cards: two exposed and one hidden. Running the program with 34 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 34 2/19/15 9:22 AM COLUMNS WORK THE SHELL More important, it also means that the game can identify situations where there’s no point in betting, like when a 7 of diamonds and 8 of clubs are dealt out. just that code results in this: $ sh acey-deucey.sh Ive dealt: Ace of Hearts helpful, particularly knowing that the dealCards function ensures that the two cards displayed are in order of increasing rank, which means that this is a darn helpful addition: Queen of Diamonds $ splitValue=$(( $rank2 - $rank1 )) There’s not much to do yet, because there’s no game logic, so let’s add some. More important, it also means that the game can identify situations where there’s no point in betting, like when a 7 of diamonds and 8 of clubs are dealt out. There are no cards that can be

between them. This is added with a simple test: Turning the Code into a Playable Game To start, let’s initialize and deal out the cards. With the highly mnemonic function names already assigned, it’s QUITE READABLE if [ $splitValue -le 1 ] ; then echo "No point in betting when you cant win!" continue initializeDeck fi shuffleDeck dealCards echo "Do you think the next card will be between? (y/n/q) " read answer This is good for a start, but as I mentioned earlier with the math discussion, it can be a bit more The third card already has been “dealt” within the function dealCards , its rank calculated (as $rank3 ) and its display name set (as $cardname3 ). So, the test to see if the new card is or isn’t between the two existing ranks is the next section of the WWW.LINUXJOURNALCOM / MARCH 2015 / 35 LJ251-March2015.indd 35 2/19/15 9:22 AM COLUMNS WORK THE SHELL So you can pick three cards randomly out of the deck, you can calculate their ranks

and display names, and you can prompt the user to guess whether the next card will or won’t be between the two, then test to see if they were right. CODE REQUIRED AND IT TOO IS EASY won=$(( $won + 1 )) else if [ $rank3 -gt $rank1 -a $rank3 -lt $rank2 ] ; then # winner! winner=1 echo "Bad betting strategy. You lose" fi else winner=0 fi So you can pick three cards randomly out of the deck, you can calculate their ranks and display names, and you can prompt the user to guess whether the next card will or won’t be between the two, then test to see if they were right. What’s left? Scoring. And, that’s done with the $won variable, which is incremented in a conditional statement that appears immediately after the test to see if the third card is a $winner or not: if [ $winner -eq 1 -a "$answer" = "y" ] ; then echo "You bet that it would be between the two and it is. You WIN!" won=$(( $won + 1 )) elif [ $winner -eq 0 -a "$answer"

= "n" ] ; then You’ll notice that in this implementation of Acey-Deucey, I’m allowing the player to win if he or she bet the card won’t be between the two, and it turns out that it isn’t. This is probably too generous, because all you need to do is pick the more likely scenario, which is to say any situation where the spread is six cards or less (like at the very beginning of this article). 3TILL ITS NOT 6EGAS OR !TLANTIC #ITY ITS just a shell script, right? So I’ll be nice. If you’d rather not offer that option, simply change the message in the first elif conditional code block and skip incrementing the $won variable. All that’s left to do is to wrap the entire code block in a big loop that’ll run FOREVER AND USE THAT STANDARD TECHNIQUE of shell script programmers worldwide: echo "You bet that it would not be between the two and it isnt. You WIN!" while [ /bin/true ] ; do 36 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 36 2/19/15

9:22 AM COLUMNS WORK THE SHELL You probably wondered why /bin/true existed in Linux, didn’t you? So that’s the first line of the main code block, and let’s increment the $games variable in the last line of the block: The spread is 3. Do you think the next card will be between them? (y/n/q) n I picked: 9 of Hearts You bet that it would not be between the two and it isnt. You WIN! Ive dealt: games=$(( games + 1 )) Ace of Hearts 7 of Spades But there’s one more fragment needed, and that’s the test to see if the user guessed that the third card would or would not be between the two displayed cards, or if the user QUIT THE GAME )N THE LATTER SITUATION it’s time to display some stats. That’s easy enough, and it turns out that you can just leave $answer alone for the yes/no answer: The spread is 6. Do you think the next card will be between them? (y/n/q) y I picked: 3 of Spades You bet that it would be between the two and it is. You WIN! Ive dealt: 7 of Spades 10 of

Spades The spread is 3. Do you think the next card will be between them? (y/n/q) q You played 2 games and won 2 times. if [ "$answer" = "q" ] ; then $ echo "You played $games games and won $won times." exit 0 fi )N FACT YOULL NEVER QUIT THE GAME by falling out of the while loop, but that makes sense since the conditional test of /bin/true is, well, always true. Stitch all these fragments together and you have a game, by George! ! PERFECT SCORE .ICE ,AS 6EGAS here I come! Q Dave Taylor has been hacking shell scripts for more than 30 yearsreally. He’s the author of the popular Wicked Cool Shell Scripts (and just completed a 10th anniversary revision to the book, coming very soon from O’Reilly and NoStarch Press). He can be found on Twitter as @DaveTaylor and more generally at his tech site http://www.AskDaveTaylorcom $ sh acey-deucey.sh Ive dealt: 6 of Hearts 9 of Clubs Send comments or feedback via http://www.linuxjournalcom/contact or to

ljeditor@linuxjournal.com WWW.LINUXJOURNALCOM / MARCH 2015 / 37 LJ251-March2015.indd 37 2/19/15 9:22 AM COLUMNS HACK AND / Libreboot on an X60, Part I: the Setup KYLE RANKIN Find out what Libreboot is and why you should dust off that old ThinkPad and give it a fresh BIOS. Recently I wrote a review for the Linux Journal Web site on the Purism Librem 15 laptop (http://www.linuxjournalcom/ content/purism-librem-15-review). The goal of this laptop is to provide a piece of modern hardware that can run 100% free software not just for the OS, but also all device drivers and firmware up to and including the BIOS. At the time I’m writing this, the last major sticking point along those lines for the PROJECT IS THE )NTEL -ANAGEMENT %NGINE a proprietary piece of firmware that is REQUIRED TO BOOT UP MODERN SYSTEMS )N that review, I wrote the following: It turns out it’s rather difficult to have a fully free software laptop. %VEN IF YOU CAN PICK HARDWARE THAT can use free software

drivers, there’s still that pesky BIOS. While coreboot and libreboot are great free software BIOS implementations, to get it on MANY LAPTOPS REQUIRES HARDWARE BIOS chip flashing with pomona clipsthe kind of thing I wasn’t ready to brick a laptop to try. Like other privacy advocates, I turned to the old ThinkPad X60 laptop series. While it’s old, underpowered and has a low-res screen by today’s standards, the keyboard is great and more important, you could flash its BIOS with coreboot or libreboot from within Linux itselfno hardware HACKING REQUIRED 3O THATS WHAT ) DID Although the Purism 15 laptop seems to be a viable choice for those who want a free software laptop, at the time of this writing, the crowdfunding campaign is still in process, and even after it completes, 38 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 38 2/19/15 9:22 AM COLUMNS HACK AND / I’ve been able to find used ThinkPad X60 laptops on auction sites as cheap as $30, so if you are

willing to live with some of the limitaions of hardware that old, it is an inexpensive route to a decent machine that runs only free software. it will take some time until they ship. Plus, a new laptop like that doesn’t come cheap, and many people who may want a laptop that runs 100% FREE SOFTWARE MAY NOT HAVE   to spend on it. I’ve been able to find used ThinkPad X60 laptops on auction sites as cheap as $30, so if you are willing to live with some of the limitations of hardware that old, it is an inexpensive route to a decent machine that runs only free software. The first time I attempted to flash an X60 with coreboot, it was one of the more difficult things I’d done with Linux to the point that I wasn’t ever planning on writing it up in Linux Journal. More recently, I tried again, only this time with Libreboota coreboot BIOS distribution that has all of the proprietary software removed. The process was greatly simplified and automated to the point where I feel

relatively comfortable recommending others try it (with a few caveats I’ll explain later). In my next couple articles, I’m going to walk through the journey that brought me to the X60 running Libreboot that I’m using to type this column. In this first part, I discuss the setup, including what Libreboot is, what hardware it currently supports and some of the risks around flashing your BIOS. If I haven’t scared you off by the end of this article, in future articles, I’ll cover how to download Libreboot and verify its integrity, how to flash the BIOS itself in detail with sample script output and how to modify the default GRUB bootloader. If you can’t wait until next month, a lot of my process is based on the excellent guide provided at https://github.com/ bibanon/Coreboot-ThinkPads/wiki/ ThinkPad-X60. Free as in BIOS To understand Libreboot, it helps to understand coreboot first. Coreboot is an open-source BIOS replacement. With coreboot, you can replace a proprietary BIOS

with open-source WWW.LINUXJOURNALCOM / MARCH 2015 / 39 LJ251-March2015.indd 39 2/19/15 9:22 AM COLUMNS HACK AND / Libreboot is a custom distribution of coreboot that removes all proprietary software from the BIOS. software on supported hardware with a minimal amount of proprietary firmware included to support things like video hardware in the BIOS or the )NTEL -ANAGEMENT %NGINE ON NEWER hardware. Coreboot doesn’t currently support all hardware out there, although the list continues to grow, and you might be surprised to know that Chromebooks ship with coreboot by default. To install coreboot on much of the supported hardware, you must use external hardware including a connector like an 8-pin Pomona clip to reflash the BIOS chip. That’s pretty intense for a lot of people, but fortunately, some hardware including the X60, X60s, X60 tablet and T60 can be flashed completely in software. When I first attempted to flash an X60 with coreboot a few months ago, the process involved

disassembling the laptop to inspect the underside of the motherboard with a magnifying glass so I could determine which of two BIOS chip types I had. I used that information to hand-patch the flashrom software with custom code and compiled a special version just to unlock my BIOS. Then I downloaded, configured and compiled a custom coreboot BIOS image for my laptop and went through a two-phase flash. In the end, I got it working; however, I needed to strip out and include the proprietary video firmware from my proprietary BIOS to get any video at boot timeuseful when you want to select between hard drive and USB boot. Libreboot is a custom distribution of coreboot that removes all proprietary software from the BIOS. Instead of proprietary BIOS boot selector, for instance, Libreboot boots straight into its own GRUB menu that you can use to load your own underlying OS. In addition, Libreboot has automated a lot of the difficult processes around installing coreboot and provides custom

scripts and pre-build ROMs for its officially supported hardware. But, why would you want a free software BIOS? For those who fully support the Free Software Foundation and the principles of free software, you don’t need any further justification. Although I have traditionally taken a more pragmatic approach to the free vs. open-source software debate, I’ve recently been more motivated to seek out free software whenever I can find it as I 40 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 40 2/19/15 9:22 AM COLUMNS HACK AND / explain in my Librem 15 review: In the past, I didn’t care all that much if I had to use a binary blob to get a wireless card or video card working as long as it worked, and I definitely never cared that my BIOS was proprietary software. Then the Snowden leaks happened. The sheer depth and breadth of the loss of privacy motivated me to step up my game in terms of overall security and focus on privacy. In the past it would seem rather

paranoid to think that there might be some sort of NSA-sanctioned spyware in a binary blob, firmware, or the BIOS. After the Snowden leaks and the SUBSEQUENT DISCLOSURES ABOUT THE ANT catalog, these things stopped seeming so far-fetched. I found myself leaning more toward the Stallman camp. One of the only ways to be truly sure that you don’t have a backdoor on your system is to be able to see the source code for all of it from the browser plugins to the kernel drivers all the way to the BIOS. Supported Hardware Due to the fact that Libreboot avoids any proprietary firmware in the BIOS, its hardware support is somewhat limited. Among other reasons, this is due to the fact that modern Intel HARDWARE REQUIRES THE PROPRIETARY )NTEL -ANAGEMENT %NGINE FIRMWARE EVEN TO boot. Although you may be able to get Libreboot to work on other hardware, at this point, only a few laptops are listed on its hardware compatibility list (http://libreboot.org/docs/hcl/ index.html#supported list) as

officially supported: Q Lenovo ThinkPad X60/X60s Q Lenovo ThinkPad X60 Tablet Q Lenovo ThinkPad T60 Q Apple MacBook1,1 Q Apple MacBook2,1 You may find one major thing in common with all the laptops on this list: they are old. In most cases, we are talking about 32-bit Intel Core $UO PROCESSORS OR  BIT #ORE  $UOS in some cases (and the T60’s CPU CAN BE REPLACED WITH A  BIT #05 apparently). That said, the X60 is a decent piece of hardware with a solid keyboard and decent battery life, even if the CPU is slow and the screen resolution is low by today’s standards. %VEN ON THIS LIST OF SUPPORTED WWW.LINUXJOURNALCOM / MARCH 2015 / 41 LJ251-March2015.indd 41 2/19/15 9:22 AM COLUMNS HACK AND / hardware there are some exceptions. Although all X60s are supported, only T60s that use Intel GPUs are supported, and those with ATI GPUs are not. The Libreboot hardware compatibility page has more information to help you figure out what’s supported and what isn’t. The page also

lists recommended Wi-Fi chipsets that are known to work well with Libreboot and Linux in general, AS THEY DONT REQUIRE ANY PROPRIETARY binary blobs to function. Risky Business If it doesn’t already go without saying, reflashing the BIOS on your laptop with custom software is risky! Although I’ve had success so far flashing a couple different X60s, I did temporarily brick one laptop when I got fancy and tried an initial flash with one of my own custom ROMs instead of one provided by Libreboot. For the most part, the process is straightforward and automated, but as you’ll see in my follow-up article that describes each step, many of the automated scripts call other software that output some pretty scary warnings and errors during the process that you are supposed to ignore. There are two primary ways you can brick your laptop during the process. First, you could have a bad flash during the initial bootstrapping flash phase. If that happens but you were using one of the

Libreboot-supplied ROMs, all you should have to do is shut off the machine, unplug the CMOS battery for a few seconds, reconnect it and power on your machine to get back to the original BIOS. If you flash during the initial bootstrapping phase with a custom ROM like I tried one time, lose power during the process, attempt this on incompatible hardware or otherwise encounter a worst-case scenario, you could end up with a completely unbootable machine. Because you can’t boot back to your OS, you can’t attempt to reflash, so you are stuck with a bricked laptop unless you buy hardware that can flash your BIOS chip, such as a BusPirate or a Raspberry Pi running custom software. That said, if you have that hardware, wire it properly and you remembered to back up your original BIOS first, you should be able to restore your laptop to normal. Although so far I’ve been successful when I’ve stuck strictly to the directions, there is still a possibility you will brick your laptop, so if

you are particularly attached to your laptop and can’t risk it being OUT OF SERVICE WHILE YOU ACQUIRE hardware flashing tools, you may 42 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 42 2/19/15 9:22 AM COLUMNS HACK AND / want to reconsider going down this road. Again, you can get used X60s relatively cheap on-line if you shop around, so if you are concerned, you may want to try this first with a sacrificial machine. AND REQUIRES A NUMBER OF UNUSUAL steps, most of the hard work already has been done for you, and in the end you’ll have a trusted machine without any proprietary firmware. Q Kyle Rankin is a Sr. Systems Administrator in the San Francisco Conclusion Well, if I haven’t scared you off yet, I hope you check out my next column in this series where I jump right into step-by-step instructions on how to flash an X60 with Libreboot. Although THE PROCESS ISNT QUITE AS SIMPLE AS updating a traditional proprietary BIOS Bay Area and the author of a number

of books, including The Official Ubuntu Server Book, Knoppix Hacks and Ubuntu Hacks. He is currently the president of the North Bay Linux Users’ Group. Send comments or feedback via http://www.linuxjournalcom/contact or to ljeditor@linuxjournal.com LINUX JOURNAL on your Android device Download the app now on the Google Play Store www.linuxjournalcom/android For more information about advertising opportunities within Linux Journal iPhone, iPad and Android apps, contact John Grogan at +1-713-344-1956 x2 or ads@linuxjournal.com LJ251-March2015.indd 43 2/19/15 9:22 AM COLUMNS THE OPEN-SOURCE CLASSROOM The Teeny Tiny $20 Tablet SHAWN POWERS What’s better than a pocket-sized Android tablet? One for $20. For reasons other than “which do you like better”, my cell phone is an Apple iPhone. Mainly it’s because the rest of my family members use Apple products, and I want to be able to fit into their environment. With three teenage daughters, it’s nice to run “Find my

iPhone” and see why they’re running late. That leaves me with two problems. First, there’s the ridicule and teasing from my geeky friends. (You know who you are!) The second problem is that I really love Android apps for much of what I do on a day-to-day basis. My Nexus 7 is too unwieldy to carry around all the time, so I really need a tiny little Android tablet I can keep in my pocket. If the roles were flipped, I could just buy an iPod Touch and be done with it. It turns out things aren’t so simple in the Android world. I was able to find the Samsung Galaxy Player in several sizes, but not only did they cost hundreds of dollars, they also were discontinued before I ever could order one. It seems my demographic is tiny enough that it can’t support a line of devices. Thankfully, my demographic is also pretty nerdy, so with a little research and hard work, I got a better solution altogetherfor $20. My Prepaid Non-Phone The short version is that I bought a prepaid Android

phone and never activated it. That version of the story leaves out some really important and really cool details, however. My end result is a pocket-sized Android device that I can use for listening to audiobooks via Bluetooth headset, make and receive calls, play games, and sorta use for a GPS device while driving. The best part is that my uber-micro-tablet really did cost me only $20. If you’re lazy, you can just buy a prepaid phone off the shelf and never activate it. Most (but not all) will allow you to cancel the 44 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 44 2/19/15 9:22 AM COLUMNS THE OPEN-SOURCE CLASSROOM If you do some research and don’t mind a little hard work, however, you can get a cheap Android device that does everything you want without any nag screens or limitations. activation screen and use the device without cell service. If you do some research and don’t mind a little hard work, however, you can get a cheap Android device that does

everything you want without any nag screens or limitations. I describe my process here, and if it sounds like something interesting, you can do the same. My Requirements I wanted my new anti-iPod to be every bit as useful as the Samsung Galaxy Player would have been. Here’s what I expected: Q ! SMALL BUT NICE QUALITY SCREEN I didn’t want a cheap plastic screen that would haze over with tiny scratches. Preferably, I wanted Gorilla Glass. Q Wi-Fi: this seems obvious, but with cheap prepaid phones, you never can tell. It’s always safest to make sure! Q At least a dual-core CPU: I didn’t want a powerhouse, but I wanted to be able to do things with the device. I wanted at least 2GB of RAM as well, but I ended up settling for 1GB. Q MicroSD expansion slot: this is vitally important, because prepaid phones generally come with absurdly small amounts of internal storage. Q Bluetooth: the main purpose of this device will be to listen to audiobooks. For that, it needs to work with

my knockoff-brand version of the Logitech HB-730. Q Must be rootable: this is as important as the MicroSD slot. 7ITH THE ADVENT OF !NDROID  you need to have a rooted device in order for applications to be able to write to the SD card. I personally think it’s about the dumbest “feature” a new version of Android could offer, but at least with root access, it can be fixed. WWW.LINUXJOURNALCOM / MARCH 2015 / 45 LJ251-March2015.indd 45 2/19/15 9:22 AM COLUMNS THE OPEN-SOURCE CLASSROOM Q Must be affordable. I already have a phone (the iPhone), so I have to be able to convince my wife that it’s not wasteful to buy a prepaid phone I never plan to activate. (APPY WIFE  HAPPY LIFE
great battery life along with really great cameras. You currently can pick up this device for around $60, and for the hardware you get, I’m guessing Boost is losing some money on every sale. My New Non-Phone I spent a very, very long time researching what phone to purchase. Since what I was

proposing goes against everything the prepaid vendors stand for, it’s not like I could check their Web sites to see if the phones were rootable or if they’d work without activation. I considered several models: 3. LG Realm from Boost Mobile: the Realm is what I ended up buying (Figure 1). The specs are a step DOWN FROM THE 6OLT BUT ) WAS ABLE to get the device for $19.99 from "EST "UY AT THE END OF  IN THE “last chance to get Black Friday Sale Prices” sale. That sale probably still is running; it seems that’s how Black Friday sales work nowadays.  -OTOROLA -OTO FROM 6ERIZON and Boost Mobile: the Moto G is a pretty decent-looking phone, and it has a beautiful screen. Unfortunately, although it has 8GB of onboard storage, it lacks a MicroSD expansion slot. It’s also around $80, which is reasonable considering how nice of a device it is, but without that SD slot, it’s more than I was willing to pay. Phone models change all the time. Rather than make

decisions based on my findings from a few months ago, I urge you to look for the latest and greatest (or cheapest!) prepaid options out there, and make sure THEY MEET YOUR LIST OF REQUIREMENTS I can’t stress enough how important it is for the phone to be rootable though, so do at least that much research before buying one.  , 6OLT FROM "OOST -OBILE THIS phone is probably what I’d buy if I were going to buy another device right now. It checks all the boxes above, and it has really The Rooting Sometimes the hardest part of the PROCESS IS TO GET OUT OF THE h!#4)6!4% -% ./7v SCREEN 7ITH ENOUGH button pressing, I was able to put the 46 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 46 2/19/15 9:22 AM COLUMNS THE OPEN-SOURCE CLASSROOM Figure 1. Oddly, a replacement battery for this phone costs more than the phone itself. At $1999, you can’t go wrong! activation screen in the background. %VERY TIME THE PHONE BOOTED HOWEVER it had the same annoying screen

trying to force me to activate. Therefore, the very first thing I recommend doing is rooting the phone. Usually, that’s as simple as visiting http://towelroot.com from the phone’s browser and installing the tr.apk file As long as your phone is supported, it’s literally 2–3 clicks, and your phone is rooted. Then install SuperSU from the Google Play store, and your phone is ready to hack. WWW.LINUXJOURNALCOM / MARCH 2015 / 47 LJ251-March2015.indd 47 2/19/15 9:22 AM COLUMNS THE OPEN-SOURCE CLASSROOM It’s important to note that rooting a phone is not the same as installing a third-party ROM. Although it’s dead simple to root most phones, installing something like Cyanogenmod is far more difficult, and often it’s not possible even if the phone is rooted. Thankfully, once the phone is rooted, the existing ROM can be made to function a little nicer. Getting rid of the nag screens is the first obstacle in that journey. Stop the Nags! Once your phone is rooted, it’s time

to start looking for the applications that are doing all the nagging. Unfortunately, this will take some googling, some guessing and a little bit of luck. The process is itself pretty straightforward: 1. Download a root-enabled file manager app like Root Browser or something similar. 2. Figure out what app(s) are responsible for the activation nag screens. Basically, google your phone’s model along with “disable activation screen” or something like that. 3. Rename the apk files to add .bak at the end In my case, I had to rename /system/app/ LGDMSClient.apk to /system/app/ LGDMSClient.apkbak  2EBOOT THE PHONE AND SEE IF IT worked. If it did, celebrate If not, do some more googling, or just educated guessing, and try again. There is some danger here, but as long as you’re not deleting files, just renaming them, most bad guesses can be reversed. Other Anti-iPod Tweaks Once your phone is working, and you’re able to reboot it without the frustrating nag screens, head over to

the settings app. Here is where you can disable all cellular data radios. Since you’re not going to activate the phone with cell service, it will save some serious battery power if you disable the radios entirely. Depending on your model, you may have to DISABLE ,4% AND  SEPARATELY The one frustration I have is that try as I might, I’ve not been able to remove the cellular radio icon from the top of the phone (Figure 2). There are some apps in the Google Play store to remove icons, but they remove the W i-Fi icon too, and that doesn’t help me at all. Oh well, it’s a small price to pay. Plus, it’s a great way for me to keep track of 48 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 48 2/19/15 9:22 AM COLUMNS THE OPEN-SOURCE CLASSROOM Figure 2. Although the cellular radio icon bothers me, it’s the only frustration I haven’t been able to eliminate! Sprint coverage in my area. If it ever gets strong enough, I’ll probably invest in a Karma router

(http://www.yourkarmacom), which would make my new minitablet even more useful on the road. One last recommendation I have is to download one of the “SD Fix” WWW.LINUXJOURNALCOM / MARCH 2015 / 49 LJ251-March2015.indd 49 2/19/15 9:22 AM COLUMNS THE OPEN-SOURCE CLASSROOM apps from the Google Play store. 7ITH THE ADVENT OF !NDROID  THE SD card isn’t writable by apps like FolderSync, and as such, it makes managing audiobooks or MP3 files really difficult. With a rooted phone, it’s another two-click solution to make your SD card functional again. I still can’t believe Google crippled Android  LIKE THAT 4HANKFULLY ROOT ACCESS and Linux can save the day. Here’s My Number, Call Me Maybe If the cell radio icon bothers me, just imagine how much it bothers me to have a phone with microphone and speaker, but no phone service. I know, I said I wasn’t looking for a phone, but I have OCD, so that unused HARDWARE REALLY ANNOYS ME %NTER 3)0 Some phones come with a

firmware that supports Android SIP calling out of the box. Most prepaid phones, however, disable that feature because they want you to use their service. It makes sense. Thankfully, you can download a third-party app called CSipSimple and add complete Wi-Fibased SIP calling to your phone. It even integrates with the native dialer application, so you use it like a regular phone! I’m still using OOGLE 6OICE SERVICE THROUGH http://www.simonicscom, but because that ability was supposed to stop almost a year ago, I wouldn’t count on it working forever. Any SIP provider will work with CSipSimple, however, so EVEN IF THE FREE OOGLE 6OICE OPTION through Simonics stops working, you can get the prepaid phone working without paying the cellular provider. GPS! It’s hard to buy an Android device that doesn’t come with GPS built in. Although the lack of cellular radio means you can’t do real-time map downloading on the road (unless you have mobile Wi-Fi or a hotspot on your actual cell

phone), it’s simple to use your new mini-tablet as a GPS device with a little bit of planning. Google Maps allows you to download map data for specific areas locally to the device. This doesn’t work great for long trips, because grabbing all those “map sections” is tedious, but for short trips to unknown locations it works well. There are also several off-line GPS apps available in the Google Play store, and although most cost money, they’re cheaper than buying a standalone GPS unit at the store. If I’m being completely honest, I still use a Garmin GPS for long trips, but that’s probably because I’m old and don’t always trust technology to work as expected. 50 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 50 2/19/15 9:22 AM LJ251-March2015.indd 51 2/19/15 9:22 AM COLUMNS THE OPEN-SOURCE CLASSROOM What Else? For me, having a tiny Android device that lets me sync audiobooks with FolderSync and play with the Listen app is all I could want and more.

That doesn’t mean I couldn’t think of more things to do with a cheap Android device, however. I actually bought another LG Realm when they were on sale, and I have lots of plans for it things like: Q A Plex player for watching movies. Q A really cheap IP camera (birdcam!) using the IPWebcam app. Q Music player for the bathroom counter, connected to speakers. Q Tiny gaming device for bored children who visit. Q Cheap Skype/Hangout device to give someone I want to keep in contact with. Really, I’d do anything you can do with any other Android device, but without the guilt of spending a ton of money. Having a $20 Android device capable of doing so many things really makes it easy to come up with fun projects. It’s also nice to be able to mount one in the car and just leave it there. With FolderSync, it will download any new media when you’re parked in the garage, and you’ll never have to take your phone out of your pocket! )M REALLY QUITE HAPPY 3AMSUNG discontinued its

Galaxy Player devices. I worry that if they were available, I might have purchased one. I’m so much happier with my $20 tablet than I would have been with a $300 media player. If you have a need for another Android device in your life, but don’t want to spend a fortune, I urge you to check out the prepaid phone options out there. It’s surprising what $20 will buy! Q Shawn Powers is the Associate Editor for Linux Journal. He’s also the Gadget Guy for LinuxJournal.com, and he has an interesting collection of vintage Garfield coffee mugs. Don’t let his silly hairdo fool you, he’s a pretty ordinary guy and can be Q Surprisingly affordable alarm clock for my nightstand. reached via e-mail at shawn@linuxjournal.com Or, swing by the #linuxjournal IRC channel on Freenode.net Q Universal XBMC/Kodi remote for all our televisions. Q Wi-Fi testing tool (WiFi Analyzer). Send comments or feedback via http://www.linuxjournalcom/contact or to ljeditor@linuxjournal.com 52 / MARCH

2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 52 2/19/15 9:22 AM LJ251-March2015.indd 53 2/19/15 9:22 AM NEW PRODUCTS Gumstix Inc.’s Geppetto Gumstix Inc. is so proud of the embedded systems it designed with its home-grown Geppetto design tool that it wants the wider world to enjoy similar benefits. Gumstix calls the new Geppetto 2.0 the most advanced version of the company’s on-line build-to-order tool for designing custom-embedded Linux systems. This new iteration of Geppetto introduces Tux-approved recommended mappings for buses, ensuring optimal compatibility between customer-created hardware and standard Linux images. In addition, version 20 offers an expanded module selection, improved dimensioning, faster UI and video tutorials. As part of the Geppetto announcement, UMSTIX ALSO ANNOUNCED THE EPPETTO DESIGNED !ERO#ORE4-  -ICRO !ERIAL 6EHICLE #ONTROL "OARD AND THE EPPETTO DESIGNED 0EPPER $6) $ SINGLE BOARD COMPUTER http://www.gumstixcom

Investintech.com’s Able2Extract PDF Converter )TS NOT A STRETCH TO CALL )NVESTINTECHCOMS !BLE%XTRACT  0$& #ONVERTER THE h3WISS ARMY KNIFEv OF 0$& CONVERTERS .OT ONLY IS !BLE%XTRACT ABLE to convert PDFs to a wide range of formats accurately, but it also features THE UNIQUE ABILITY TO WORK ACROSS ,INUX 5BUNTU AND &EDORA -AC /3 8 AND 7INDOWS PLATFORMS )NVESTINTECHCOM NOTES THE ABILITY OF !BLE%XTRACT to maintain intact all aspectsimages, colors, formatting and fonts regardless of file format. Supported formats include converting PDF to /PEN/FFICEORG -3 /FFICE !UTO#!$ %XCEL AND COMMONLY USED IMAGE formats. The upgrade version 9 adds secure PDF creation, improved custom 0$& TO %XCEL CONVERSION AND AN IMPROVED 5) AND OVERALL USER EXPERIENCE http://www.investintechcom 54 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 54 2/19/15 9:22 AM NEW PRODUCTS Linutop XS Until the era of the Linutop computer, the word minuscule has not been a common

descriptor for a full-fledged PC. That word nevertheless hits the NAIL SQUARELY ON THE HEAD TO DESCRIBE THE NEW Linutop XS, a truly tiny Linux computer designed to reduce TCO from shipping to deployment, operation and maintenance. As Linutop’s smallest and most energy-efficient computer to date, the Linutop XS weighs a mere 3.3 ounces (92 g), measures about the size OF A TYPICAL PLAYING CARD AND OPERATES ON ONLY  6OLTS AND  7ATTS ,INUTOP SAYS that the Linutop XS comes loaded with Debian Weezy and ready-to-use software, including Libre Office and Linutop Kiosk, making it an ideal system for a wide range of applications in business, government, education and the home. http://www.linutopcom JetBrains’ Upsource The idea for JetBrains’ new team collaboration tool for developers, called Upsource, originally came from the intention to make a TOTALLY DIFFERENT TOOL )NTELLI* )$%! available from both the desktop and the Web. The final result is Upsource 10, a new Web-based team

collaboration tool that helps developers read, browse and review code maintained in Git, Mercurial, Subversion and/or Perforce repositories. Both a repository browser and a code-review tool, Upsource 1.0 provides instant read access to code developed throughout an ORGANIZATION AND HELPS IMPROVE CODE QUALITY BY ENABLING EASY CODE REVIEW *ET"RAINS ADDS THAT THANKS TO PLATFORM SHARING WITH THE )NTELLI* )$%! )$% FOR AVA AVA TEAMS enjoy an additional advantage. Upsource boasts in-depth knowledge of Java code and is able to execute server-side static code analysis on Java projects, as well as provide code-aware navigation and smart search for code usages. http://www.jetbrainscom/upsource WWW.LINUXJOURNALCOM / MARCH 2015 / 55 LJ251-March2015.indd 55 2/19/15 9:22 AM NEW PRODUCTS Corsair Flash Voyager Slider Series X1 and X2 4HE NEW &LASH 6OYAGER 3LIDER 8 AND 8 FAMILIES OF 53"  &LASH drives expand Corsair’s already formidable arsenal of memory products.

Combining the speed of USB 30 with the functionality of A CAP LESS 53" DRIVE THE &LASH 6OYAGER 3LIDER 8 AND 3LIDER 8 SERIES share a sleek, glossy design that allows the USB cap to slide back conveniently into the drive housing, says Corsair. The company added that the Slider 8 IS AVAILABLE IN " " " " AND " CAPACITIES AND THANKS TO ITS USB 3.0 interface, is able to reach read speeds of up to 130MB/s Meanwhile, Slider X2 knocks the performance up a level with read speeds of 200MB/s in capacities of " " " AND " "OTH #ORSAIR DRIVE FAMILIES ARE COMPATIBLE WITH ,INUX Windows and Mac OS X, and they also are fully USB 2.0-backward compatible http://www.corsaircom Regina O. Obe and Leo S Hsu’s PostGIS in Action, 2nd ed. (Manning Publications Co.) Hybrid GIS and Linux geeks know that the open-source 0OST)3 GIVES SUPPORT FOR GEOGRAPHIC OBJECTS TO 0OSTGRE31, allowing the relational database to serve as the back

end for ArcGIS, GRASS GIS and other geospatial programs. The new 2nd edition of PostGIS in Action from Regina O. Obe AND ,EO 3 (SU TEACHES READERS OF ALL LEVELS TO WRITE SPATIAL QUERIES THAT SOLVE real-world problems. Obe and Hsu start by getting readers’ feet wet with a background in vector-, raster- and topology-based GIS, followed by a tutorial IN ANALYZING VIEWING AND MAPPING DATA 2EADERS LEARN HOW TO OPTIMIZE QUERIES for maximum speed, simplify geometries for greater efficiency, analyze rasters, vectorize rasters, better manage data utilizing topologies and create custom FUNCTIONS 4HE BOOK COVERS 0OST)3  AND  0OSTGRE31,   AND  features and shows how to integrate PostGIS with other GIS tools. http://manning.com 56 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 56 2/19/15 9:22 AM NEW PRODUCTS Mahesh Venkitachalam’s Python Playground (No Starch Press) Putting the subtitle Geeky Weekend Projects for the Curious Programmer onto a book is a sure way

to charm one’s way onto these geek-friendly Linux Journal pages. The main title of said book is Python Playground A NEW BOOK FROM -AHESH 6ENKITACHALAM and irreverent publisher No Starch Press. No Starch describes the book as “a collection of fun programming projects that will inspire you to new heights as a Pythonista”. Readers will learn to use Python for all kinds of playful purposesfor example, to manipulate images, build simulations and interact with hardware using Arduino and Raspberry Pi. As readers work through each project, they power up their programming skills and learn how to leverage external libraries for specialized tasks, how to break problems into smaller, solvable pieces and how to translate an algorithm into code. The fun projects include an autostereogram generator, an ASCII art maker, a Conway’s Game of Life simulator, a ray casting volume renderer and an Arduino rig. http://www.nostarchcom Deciso OPNsense Firewall $ECISO "6 IS A .ETHERLANDS BASED

MANUFACTURER OF NETWORKING EQUIPMENT THAT DEVELOPED AND RECENTLY RELEASED /0.SENSE A NEW open-source firewall that reportedly “combines the best of opensource and closed-source firewalls”. Deciso adds that OPNsense brings the rich feature set of commercial offerings with the benefits of open and verifiable sources, combined with a simple, two-clause BSD license. The latter permits companies to create a branded firewall based on OPNsense, extend its features, or even create a fork and build upon the same codebase. Key features of OPNsense include load balancing, high availability and captive portal. The easy-to-use Bootstrap-based GUI makes configuring and managing the firewall a comfortable task for administrators. The kicker, boasts Decisio, is that all sources and build tools are freely available without special clauses and without licensing costs. The company also puts a great deal of value on the community surrounding OPNsense, which it says will give users, developers and

businesses a friendly, stable and transparent environment. http://www.opnsenseorg and http://wwwdecisocom Please send information about releases of Linux-related products to newproducts@linuxjournal.com or New Products c/o Linux Journal, PO Box 980985, Houston, TX 77098. Submissions are edited for length and content WWW.LINUXJOURNALCOM / MARCH 2015 / 57 LJ251-March2015.indd 57 2/19/15 9:22 AM FEATURE Using Hiera with Puppet USING HIERA WITH PUPPET A GUIDE TO USING HIERA WITH PUPPET, SEPARATING CODE FROM DATA AND ENCRYPTING PASSWORDS AND CERTIFICATES. SCOTT LACKEY 58 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 58 2/19/15 9:22 AM W ith Hiera, you can externalize your systems’ configuration data and easily understand how those values are assigned to your servers. With that data separated from your Puppet code, you then can encrypt sensitive values, such as passwords and keys. Separating code and data can be tricky. In the case of configuration management,

there is significant value in being able to design a hierarchy of dataespecially one with the ability to cascade through classifications of servers and assign one or several options. This is the primary value that Hiera providesthe ability to separate the code for “how to configure the /etc/ntp.conf” from the values that define “what ntp servers should each node use”. In the most concise sense, Hiera lets you separate the “how” from the “what”. The idea behind separating code and data is more than just having a cleaner Puppet environment; it allows engineers to create more re-usable Puppet modules. It also puts your variables in one place so that they too can be re-used, without importing manifests across modules. Hiera’s use cases include managing packages and versions or using it as a Node Classifier. One of the most compelling use cases for Hiera is for encrypting credentials and other sensitive data, which I talk about later in this article. Puppet node data

originally was managed through node inheritance, which is no longer supported, and SUBSEQUENTLY THROUGH USING A PARAMSPP module subclass. Before Hiera, it was necessary to modify the params.pp module class locally within the MODULE WHICH FREQUENTLY DAMAGED the re-usability of the module. params.pp still is used in modules today, but as of Puppet version 3, Hiera is not only the default, but also the first place checked for variable values. When a variable is defined in both Hiera and a module, Hiera takes precedence by default. As you’ll see, it’s easy to use a module with params.pp and store some or all of the variable data in Hiera, making it easy to migrate incrementally. To get started using Hiera with your existing Puppet 3 implementation, you won’t have to make any significant changes or code migrations. You need only a hierarchy file for Hiera and a yaml file with a key/value pair. Here is an example of a Hiera hierarchy: hiera.yaml: :backends: - yaml :yaml:

WWW.LINUXJOURNALCOM / MARCH 2015 / 59 LJ251-March2015.indd 59 2/19/15 9:22 AM FEATURE Using Hiera with Puppet :datadir: /etc/puppet/hieradata :hierarchy: - "node/%{::fqdn}" - "environment/%{::env}/main" - "environment/%{::env}/%{calling module}" - defaults And a yaml file: /etc/puppet/hieradata/environment/prod/main.yaml: --$nginx::credentials::basic auth: password Hiera can have multiple back ends, but for now, let’s start with yaml, WHICH IS THE DEFAULT AND REQUIRES NO additional software. The :datadir: is just the path to where the hierarchy search path should begin, and is usually a place within your Puppet configuration. The :hierarchy: section is where the core algorithm of how Hiera does its key/value lookups is defined. The :hierarchy: is something that will grow and change over time, and it may become much more complex than this example. Within each of the paths defined in the :hierarchy: , you can reference any Puppet variable, even

$operatingsystem and $ipaddress , if set. Using the %{variable} syntax will pull the value. This example is actually a special hierarchical design that I use and recommend, which employs a fact assigned to all nodes called @env from within facter. This @env value can be set on the hosts either BASED ON &1$. OR TAGS IN %# OR elsewhere, but the important thing is that this is the separation of one large main.yaml file into directories named prod, dev and so on, and, therefore, the initial separation of Hiera values into categories. The second component of this specific example is a special Hiera variable called %{calling module} . 4HIS VARIABLE IS UNIQUE AND RESERVED for Hiera to indicate that the yaml filename to search will be the same as the Puppet module that is performing the Hiera lookup. Therefore, the way this hierarchy will behave when looking for a variable in Puppet is like: $nginx::credentials::basic auth First, Hiera knows that it’s looking in

/etc/puppet/hieradata/node for a file NAMED HOSTNAMEDOMAINTLDYAML and for a value for nginx::credentials::basic auth . If either the file or the variable isn’t there, the next step is to look in /etc/puppet/hieradata/ ENVIRONMENTPRODSTAGEDEV main.yaml, which is a great way to have one yaml file with most 60 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 60 2/19/15 9:22 AM of your Hiera values. If you have a lot of values for the nginx example and you want to separate them for manageability, you simply can move them to the /etc/puppet/hieradata/ ENVIRONMENTPRODSTAGEDEV nginx.yaml file Finally, as a default, Hiera will check for the value in defaults.yaml at the top of the hieradata directory. Your Puppet manifest for this lookup should look something like this: Hiera will override it. This kind of Hiera lookup is called Automatic Parameter Lookup and is one of several ways to pull data from Hiera, but it’s by far the most common in practice. You also

can specify a Hiera lookup with: modules/nginx/manifests/credentials.pp class nginx::credentials ( basic auth = hiera(nginx::credentials::basic auth), modules/nginx/manifests/credentials.pp class nginx::credentials ( basic auth = some default, ){} This class, when included, will pull the value from Hiera and can be used whenever included in your manifests. The value set here of some default is just a placeholder; Hiera will override anything set in a parameterized class. In fact, if you have a class you are thinking about converting to pull data from Hiera, just start by moving one variable from the class definition in {} to a parameterized section in (), and Puppet will perform a Hiera lookup on that variable. You even can leave the existing definition intact, because ){} These will both default to a priority lookup method in the Hiera data files. This means that Hiera will return the value of the first match and stop looking further. This is usually the only behavior you want,

and it’s a reasonable default. There are two lookup methods worth mentioning: hiera array and hiera hash . hiera array will find all of the matching values in the files of the hierarchy and combine them in an array. In the example hierarchy, this would enable you to look up all values for a single key for both the node and the environmentfor example, adding an additional DNS search path for one host’s /etc/resolv.conf To use a hiera array lookup, you must define the lookup type explicitly WWW.LINUXJOURNALCOM / MARCH 2015 / 61 LJ251-March2015.indd 61 2/19/15 9:22 AM FEATURE Using Hiera with Puppet (instead of relying on Automatic Parameter Lookup): * default :backends: * psql modules/nginx/manifests/credentials.pp :psql: :connection: :dbname: hiera :host: localhost class nginx::credentials ( :user: root basic auth = hiera array(nginx::credentials::basic auth), :password: password ){} A hiera hash lookup works in the same way, only it gathers all matching values into

a single hash and returns that hash. This is often useful for an advanced create resources variable import as well as many other uses in an advanced Puppet environment. Perhaps Hiera’s most powerful feature is the ability to pull data from a variety of back-end storage technologies. Hiera back ends are too numerous to list, but they include JSON, Redis, MongoDB and even HTTP to create a URL-driven Puppet value API. Let’s take a look at two useful back ends: Postgres and hiera-eyaml. 4O START WITH THE PSQL BACK END YOU NEED TO INSTALL THE HIERA PSQL GEM ON your Puppet master (or each node if you’re using masterless Puppet runs with Puppet apply), with a simple hiera.yaml file of: :hierarchy: * environment/%{env} You can do lookups on a local Postgres installation with a single database called hiera with a single table called config with three COLUMNS 0ATH +EY AND 6ALUE path environment/prod key nginx::credentials::basic auth value password This is extremely useful if you

want to expose your Hiera data to custom in-house applications outside Puppet, or if you want to create a DevOps Web console or reports. Storing credentials in Puppet modules is a bad idea. If you store credentials in Puppet and your manifests on an external code repository, you’re not only unable to share those manifests with developers with less-secure access, but you’re obviously exposing vital security data outside the organization, and possibly in violation of various types 62 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 62 2/19/15 9:22 AM of compliance. So how do you encrypt sensitive data in Puppet while keeping your manifests relevant and sharable? The answer is with hiera-eyaml. Tom Poulton created hiera-eyaml to allow engineers to do just that: encrypt only the sensitive string of data inside the actual file rather than encrypting the entire file, which also can be done with hiera-gpg (a very useful encryption gem but not covered in this article). To get

started, install the hiera-eyaml gem, and generate a keypair on the Puppet master: $ eyaml createkeys Then move the keys to a secure location, like /etc/puppet/secure/keys. Your hiera.yaml configuration should look something like this: hiera.yaml: --:backends: - eyaml - yaml :yaml: :datadir: /etc/puppet/hieradata :eyaml: :datadir: /etc/puppet/hieradata :extension: yaml # <- so all files can be named .yaml :pkcs7 private key: /path/to/private key.pkcs7pem :pkcs7 public key: /path/to/public key.pkcs7pem :hierarchy: - "node/%{::fqdn}" - "environment/%{::env}/main" - "environment/%{::env}/%{calling module}" * defaults To encrypt values, you need only the public key, so distribute it to anyone who needs to create encrypted values: $ eyaml encrypt -s password This will generate an encrypted block that you can add as the value in any yaml file: main.yaml: nginx::credentials::user: slackey #cleartext example value nginx::credentials::basic auth : >

#encrypted example value ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2Nn /HjZFXwcXRtTlzewJLc+/gox2IfByQRhsI/AgogRfYQKocZg IZGeunzwhqfmEtGiqpvJJQ5wVRdzJVpTnANBA5qxeA==] %DITING ENCRYPTED VALUES IN PLACE is one of the coolest features of the hiera-eyaml back end. eyaml edit opens a copy of the eyaml file in your editor of choice and automatically decrypts all of the values in the file. Here you can modify the values just as though they were plain text. When you exit the editor by saving the file, it automatically encrypts all of the modified values and saves the new file in place. You can see that the WWW.LINUXJOURNALCOM / MARCH 2015 / 63 LJ251-March2015.indd 63 2/19/15 9:22 AM FEATURE Using Hiera with Puppet unencrypted plain text is marked to allow the eyaml tool to identify each encrypted block, along with the encryption method that originally was used. This is used to make sure that the block is encrypted again only if the clear text value has changed and is encrypted

using the original encryption mechanism: nginx::credentials::user: user1 nginx::credentials::basic auth : DEC(1)::PKCS7[very secret password]! Blocks and strings of encrypted text can get rather onerous once you have more than a hundred entries or so. Because these yaml files are meant to be modified by humans directly, you want them to be easy to navigate. In my experience, it makes sense to keep your encrypted values in a separate file, such as a secure.yaml, with a hierarchy path of: :hierarchy: - "node/%{::fqdn}" - "environment/%{::env}/secure" - "environment/%{::env}/main" encrypted files in a separate repository, perhaps in a different Git repository. Only the private keys need to be protected on the Puppet master. I also recommend having separate keys for each environment, as this can give more granular control over who can decrypt different datafiles in Hiera, as well as even greater security separation. One way to do this is to name the keys

with the possible values for the @env fact, and include that in the path of the hierarchy. You’ll need to encrypt values with the correct key, and this naming convention makes it easy to tell which one is correct: :pkcs7 private key: /path/to/private key.pkcs7pem-%{::env} :pkcs7 public key: /path/to/public key.pkcs7pem-%{::env} When using Hiera values within Puppet templates, either encrypted or not, you must be careful to pull them into the class that contains the templates instead of calling the values from within the template across classesfor example, in the template mytest.erb in a module called mymodule: - "environment/%{::env}/%{calling module}" mytest.erb: This isn’t necessary, as each value is encrypted individually and can be distributed safely to other teams. It may work well for your environment, however, because you can store the . username: user1 passwd: <%= scope.lookupvar(nginx::credentials::basic auth) %> ´#dont do this . 64 / MARCH 2015 /

WWW.LINUXJOURNALCOM LJ251-March2015.indd 64 2/19/15 9:22 AM LJ251-March2015.indd 65 2/19/15 9:22 AM FEATURE Using Hiera with Puppet Puppet may not have loaded a value into mytest.erb: nginx::credentials::basic auth . yet because of the order of operations. Also, if you are using the %calling module Hiera variable, the calling module in this case would be mymodule, and not nginx, so it would not find the value in the nginx.yaml file, as one might expect. To avoid these and other issues, it’s best to import the values into the mymodule class and assign local values: username: user1 mymodule.pp: Scott Lackey is a 17-year engineering veteran and Sr. DevOps class mymodule { Engineer for Salesforce.com He’s passionate about helping passwd: <%= @basic auth %> You’re now ready to start introducing encrypted Hiera values gradually into your Puppet environment. Maybe after you separate data from your Puppet code, you can contribute some of your modules to the

PuppetForge for others to use! Q include nginx::credentials companies migrate to the cloud and mentoring prospective $basic auth = "${nginx::credentials::basic auth}" DevOps engineers. He lives in Los Angeles with his dachshund file { /etc/credentials/boto cloudwatch.cfg: Zelda. Reach him at sudosudash@gmailcom content => template ("mymodule/mytest.erb"), } And then reference the local value from the template: Send comments or feedback via http://www.linuxjournalcom/contact or to ljeditor@linuxjournal.com Resources DocsHiera 1 Overview: https://docs.puppetlabscom/hiera/1 “First Look: Installing and Using Hiera”: http://puppetlabs.com/blog/first-look-installing-and-using-hiera TomPoulton/hiera-eyaml: https://github.com/TomPoulton/hiera-eyaml dalen/hiera-psql: https://github.com/dalen/hiera-psql “Encrypting sensitive data in Puppet”: http://www.theguardiancom/info/developer-blog/ 2014/feb/14/encrypting-sensitive-data-in-puppet 66 / MARCH 2015 /

WWW.LINUXJOURNALCOM LJ251-March2015.indd 66 2/19/15 9:22 AM Interested in Site Reliability Engineering? SREcon is back! SREcon15 SREcon15EUROPE www.usenixorg/srecon15 www.usenixorg/srecon15europe MARCH  16–17,  2015   SANTA  CLARA,  CALIFORNIA,  USA MAY  14–15,  2015   DUBLIN,  IRELAND Following 2014’s inaugural sold-out conference, SREcon has expanded to two venues for 2015. If you already work in an SRE environmentor want to learn how it’s being used by many of the largest companies todaytake advantage of this rare opportunity to meet with other engineers and discuss tricks of the trade. Register today at www.usenixorg LJ251-March2015.indd 67 srecon15 lj.indd 1 2/19/15 9:23 AM 2/17/15 11:03 AM FEATURE Initializing and Managing Services in Linux: Past, Present and Future INITIALIZING AND MANAGING SERVICES IN LINUX: PAST, PRESENT AND FUTURE systemd is the new init system used by many of the top Linux distributions, but do you

know the history behind it and how we got here? Learn about the history of init systems in Linux and their UNIX legacy. Gain a better perspective about how Linux manages services and other support processes. Jonas Gorauskas 68 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 68 2/19/15 9:23 AM O ne of the most crucial pieces of any UNIX-like operating system is the init dæmon process. In Linux, this process is started by the kernel, and it’s the first userspace process to spawn and the last one to die during shutdown. During the history of UNIX and Linux, many init systems have gained popularity and then faded away. In this article, I focus on the history of the init system as it relates to Linux, and I talk about the role of init in a modern Linux system. I also relate some of the history of the 3YSTEM 6 )NIT 3YS6 SCHEME WHICH was the de facto standard for many Linux distributions for a long time. Then I cover a couple more modern approaches to system initialization,

such as Upstart and systemd. Finally, I pay some attention to how things work in systemd, as this seems to be the popular choice at the moment for several of the largest distributions. The Role of Init Init is short for initializer, and it’s both a startup manager and a session manager for Linux and other UNIXes. It’s a startup manager, because it plays a crucial role in the startup of Linux. It’s the process that creates or initializes userspace and, ultimately, all userspace processes. It also may be considered a session manager, because it takes care of many aspects of userspace and its processes once the system is up and running. The call to start this process is, in fact, hard-coded in the Linux kernel. Download the latest kernel sources and look for a function called kernel init in the file init/main.c Among the files that the Linux kernel will try to execute is /sbin/init. If Linux cannot find one of these processes, it throws a kernel panic and halts. The kernel gives

the init process an ID of 1 or PID 1. All other userspace processes are forked from init, and therefore, PID 1 claims ancestral rights to all other userspace processes. PID 1 also automatically will become the direct parent process of any userspace process that is orphaned. A Little Bit of History Now that I have set the stage for the article and given you a very basic understanding of what init is and does, I’d like to digress into a little bit of UNIX history. There has been a lot of diversity in the initialization schemes for UNIXlike operating systems over time. Two of the most important init schemes that had a historical impact on how different Linux distributions do things ARE THE RC SCHEME USED IN THE  "3$ WWW.LINUXJOURNALCOM / MARCH 2015 / 69 LJ251-March2015.indd 69 2/19/15 9:23 AM FEATURE Initializing and Managing Services in Linux: Past, Present and Future A Linux distribution implementing a SysV scheme can be in one of many distinct states in which a

predetermined number of processes may be running. AND THE 3YS6 SCHEME USED IN 3UN/3 and Solaris. 4HE  "3$ INIT SYSTEM IS PRETTY simple and monolithic. When booting, the kernel runs /sbin/init, which would spawn a shell to run the /etc/rc script. The /etc/rc script contained commands to check the integrity of hard drives and mount them, start other processes, and start the networking subsystem. This scheme was contained completely within a few scripts: namely /etc/rc, /etc/rc.local and /etc/netstart. This scheme also had no specific shutdown procedure. Init WOULD RECEIVE A 3)4%2- SIGNAL AND SEND A 3)(50 ANDOR A 3)4%2- TO its children, and after all processes exited, it would drop to single-user mode and shut down. Today, the systems that have inherited the rc initialization scheme are Free-BSD, Net-BSD and the Slackware Linux distribution. These modern systems have improved QUITE A BIT ON THE ORIGINAL  "3$ scheme and are much more modular than the original. Most other

Linux distributions have, historically, been adepts of THE 3YS6 SCHEME WHICH ORIGINALLY was implemented in AT&T UNIX and derivative systems like Solaris. System V Init A Linux distribution implementing A 3YS6 SCHEME CAN BE IN ONE OF many distinct states in which a predetermined number of processes may be running. These states are called runlevels and to get to a certain runlevel means that the system is in a certain operational stage. The meaning for each runlevel may vary based on your distribution of Linux. For example, there are a few distributions (such as Ubuntu) that use runlevel 2 to mean multi-user graphical mode with networking enabled. Others (like Fedora) use runlevel 5 to mean the same thing. )N A 3YS6 ,INUX MACHINE THE KERNEL runs /sbin/init as usual, which in turn will load parameters and execute 70 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 70 2/19/15 9:23 AM directives defined in /etc/inittab. This file defines the default runlevel for the whole

system, describes what happens when Ctrl-Alt-Del is pressed, loads keymap files, defines which terminals to spawn gettys for, spawns terminal login processes, runs the /etc/init.d/rcS script, and it also influences the order of execution of other runlevel scripts. The /etc/init.d/rcS script will put the system in a single-user mode in order to finish probing hardware, mount disks, set hostname, set up networking and so on. Take a look at /etc/rcS.d/ in a Debian 7 system for all the gory details. Next, /sbin/init will switch itself to the default runlevel to start all the system services. The default runlevel value is defined in the initdefault line of /etc/inittab. This actually translates into a call to the /etc/init.d/rc script with the parameter of 2 for the runlevel value. The rc script will then execute all of the + FOR +ILL AND 3 FOR 3TART SCRIPTS in the /etc/rc2.d/ directory These are actually links to the real scripts in /etc/init.d/ The names of the links follow the format

S##<service-name> or K##<service-name> , where the ## token is the two-digit number used to determine the order in which the script should run. Order is alphabetical, and Kill scripts execute before Start scripts. The last thing to happen is to run the /etc/rc.local script, which is where you can add custom system commands that you want to execute at startup. ! SYSTEM THAT USES THE 3YS6 SCHEME usually comes with the service program used to manage the services while the system is running. You can check on the status of a service, or all services, and start or stop a service, respectively, using the service utility: Q $ service <service> status Q $ service status -all Q # service <service> start|stop To manage the assignment of services to a particular runlevel, you can use a tool called sysv-rc-conf , which manages the setup of all links in the respective rc directories. You also can switch the runlevel of the system at any time when you use the command telinit

as a privileged user. For example, telinit 6 will REBOOT A 3YS6 SYSTEM 4HE 3YS6 SCHEME STILL IS IN USE today in Debian 7 (Wheezy) systems. However, the Debian developers will be changing the init system in version 8 to systemd. I cover systemd in more WWW.LINUXJOURNALCOM / MARCH 2015 / 71 LJ251-March2015.indd 71 2/19/15 9:23 AM FEATURE Initializing and Managing Services in Linux: Past, Present and Future The SysV scheme has been great, but it started to show its age around the time when Linux on the desktop gained a little more momentum. detail below, but first, let’s look at why we need a new init system. The Problem with System V Init 4HE 3YS6 SCHEME HAS BEEN GREAT but it started to show its age around the time when Linux on the desktop gained a little more momentum. 7HEN THE 3YS6 SCHEME ORIGINALLY was designed, computers where NOTHING LIKE THEY ARE TODAY 3YS6 was not designed to handle certain things well: shutting down services prior to shutdown. As a result, the design

was strictly synchronous, blocking future tasks until the current one had completed. This left the system unable to handle various events that were not related to the startup or shutdown of the system. Things that we take for granted today were really cumbersome to handle elegantly DURING THE HEYDAY OF 3YS6 INIT Q There was no real process Q USB devices. Q %XTERNAL STORAGE VOLUMES Q Bluetooth devices. Q The cloud. 4HE 3YS6 SCHEME WAS DESIGNED for a world that was static and slow moving. This init scheme originally was responsible only for bringing the system into a normal running state after power on or gracefully supervisionfor example, dæmons were not automatically restarted when they crashed. Q There was no real dependency checking. The order of script naming determined the order in which they were loaded. Q The addition or removal of USB drives and other portable storage/ network devices while the machine was running was cumbersome and OFTENTIMES REQUIRED A REBOOT 72 /

MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 72 2/19/15 9:23 AM Q There were no facilities to discover and scan for new storage devices without locking the system, especially when a disk might not even power on until it was scanned. Q There were no facilities to load firmware for a device, which may have needed to occur after it was detected but before it was usable. Inevitably, around the 2005/2006 time frame, several alternative efforts tried to fix all the issues with THE 3YS6 SCHEME "UT THE EFFORT THAT looked most promising during that time was the Upstart init project sponsored by Canonical. Upstart To be sure, Upstart init doesn’t share ANY CODE WITH THE 3YS6 INIT SCHEME but it’s rather a superset of it, providing a good degree of backwardcompatibility. The main departure FROM THE TRADITIONAL 3YS6 WAY OF DOING things is that Upstart implements an event-driven model that allows it to respond to milestones asynchronously as they are reached. Upstart also

implements the concept of jobs, which are described by the files under ETCINIT CONF AND WHOSE PURPOSE is to execute a script section that spawns a process. As such, system initialization can be expressed as a consecutive set of “spawn process X when event Y occurs” rules. *UST LIKE IN THE 3YS6 SCHEME the Linux kernel gives control to Upstart when it executes the Upstart implementation of /sbin/init. At this point, things may work a little differently depending on your distribution of Linux. For Red Hat %NTERPRISE ,INUX 2(%,  USERS you’ll still find a file at /etc/inittab, but the sole function of this file is to set the default runlevel for the system. If your distribution is one of the Ubuntu derivatives, /etc/inittab doesn’t even exist anymore, and the default runlevel is set in a file called /etc/init/rc-sysinit.conf instead The Upstart version of /sbin/init will emit a single event called startup, which triggers the rest of the system initialization. There are a few

jobs that specify the startup event as their start condition, the most notable of which is mountall , which mounts all filesystems. The mountall job then triggers various other events related to disk and filesystem initialization. These events, in turn, trigger the udev kernel device manager to start, and it emits the event that starts the networking subsystem. This is when one of the most critical WWW.LINUXJOURNALCOM / MARCH 2015 / 73 LJ251-March2015.indd 73 2/19/15 9:23 AM FEATURE Initializing and Managing Services in Linux: Past, Present and Future jobs is triggered by Upstart. This job is called rc-sysinit , which has a start dependency on the filesystem and network-up events. The role of this job is to bring the system to its default runlevel. It executes the command telinit <runlevel> to achieve this. The telinit command then emits the runlevel event, which causes many other jobs to start. This includes the /etc/init/rc.conf job, which implements A COMPATIBILITY

LAYER FOR THE 3YS6 INIT scheme. It executes /etc/initd/rc <runlevel> and determines if a /etc/rc#.d/ directory exists for the current runlevel, executing all scripts in it. In Upstart-based systems, such AS 5BUNTU AND 2(%,  YOU CAN use the tools sysv-rc-conf or chkconfig , respectively, to manage the runlevel of different services. You also can manage jobs via the initctl utility. You can list all jobs and their respective start and stop events with the command initctl show-config . You also can check on job status, list available jobs and start/stop jobs with the following commands, respectively: Q $ initctl status <job> Q $ initctl list Q # initctl start|stop <job> The Upstart scheme has been used in popular distributions of Linux, such AS &EDORA FROM VERSIONS  UP TO  THE 2(%,  SERIES AND 5BUNTU SINCE version 6.10 to present But for all the flexibility that Upstart init brings to Linux, it still falls short in a few fundamental ways: Q It ignores the

system state between events. For instance, a system has a power cord plugged in, then the system runs on AC power for a while, and then the user unplugs the power cord. Upstart focuses on each event above as a single discrete and unrelated unit, instead of tracking the chain of events as a whole. Q The event-driven nature of the system turns the dependency chain on its head. Instead of doing the absolute minimum amount of work needed to get the system to a working state, when an event is triggered, it executes all jobs that could possibly follow it. For example, just because networking has started, it doesn’t mean that NFS also should start. As a matter of fact, the opposite is the correct order of things: when a user REQUESTS ACCESS TO AN .&3 SHARE the system should validate that networking is also up and running. 74 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 74 2/19/15 9:23 AM The main design goals of this init scheme are, according to Lennart Poettering,

lead developer of systemd, “to start less, and to start more in parallel”. Q The dependency chain is still present. Although many more things happen in parallel in Upstart, the user has to port the original script SEQUENCE FROM 3YS6 INIT TO A SET of event trigger action rules in the CONF FILES IN ETCINIT &URTHERMORE because of the spanning tree structure of the event system, it is a real nightmare to figure out why something happened and what event triggered it. There is another init scheme whose purpose is to address the issues listed above. systemd systemd is the latest milestone on the road to init system nirvana. The main design goals of this init scheme are, according to Lennart Poettering, lead developer of systemd, “to start less, and to start more in parallel”. What that means is that you execute only that which is absolutely necessary to get the system to a running state, and you run as much as possible at the same time. To accomplish these goals, systemd

aims to act against two major trouble spots of previous init schemes: the shell and parallelism. The main executable for systemd, /lib/systemd/ systemd, performs all calls that originally were present in scripts, thus eliminating the need to spawn a shell environment. What about the call to /sbin/init that’s hard-coded in the Linux kernel? It’s still there in the form of a symbolic link to /lib/systemd/systemd. To address parallelism, you need to remove the dependency chain between the various services or at least make it a secondary concern. If you look at the problem at its most fundamental level, the dependency between the various services boils down to one thing: having a socket available for the processes to communicate among themselves. systemd creates all sockets first and then spawns all processes in parallel. For example, services that need to write to the system log need to wait for the WWW.LINUXJOURNALCOM / MARCH 2015 / 75 LJ251-March2015.indd 75 2/19/15 9:23 AM

FEATURE Initializing and Managing Services in Linux: Past, Present and Future /dev/log socket to become available, but as soon as it is available, these services can start. Therefore, if systemd creates the socket /dev/log first, then that’s one less dependency that blocks OTHER SERVICES %VEN IF THERE IS NOTHING to receive messages at the other end of the socket, this strategy still works. The kernel itself will manage a buffer for the socket, and as soon as the receiving service starts, it will flush the buffer and handle all the messages. The ideas above are not new or revolutionary. They have been tried before in projects like the xinetd superserver and the launchd init scheme used in OS X. systemd does introduce the new concepts of units and targets. A target is analogous to a runlevel in previous schemes and is composed of several units. systemd will execute units to reach a target. The instructions for each unit reside in the /lib/systemd/system/ directory. These files use a

declarative format that looks like a Windows INI file. The most common type of these units is the service unit, which is used to start a service. The sshdservice file from Arch Linux looks like this: [Service] ExecStart=/usr/bin/sshd -D ExecReload=/bin/kill -HUP $MAINPID KillMode=process Restart=always [Install] WantedBy=multi-user.target This format is really simple and really portable across several different distributions. There are other types of unit files that describe a system, and they are socket, device, mount, automount, swap, target, path, timer, snapshot, slice and scope. Going into all of them in detail is beyond the scope of this article; however, I want to mention one thing: target is a special type of unit file that glues the other types together into a cohesive whole. For example, here are the contents of basic.target from Arch Linux: [Unit] Description=Basic System Documentation=man:systemd.special(7) Requires=sysinit.target Wants=sockets.target timerstarget

pathstarget [Unit] Description=OpenSSH Daemon Wants=sshdgenkeys.service ´slices.target After=sysinit.target socketstarget timerstarget ´paths.target slicestarget After=sshdgenkeys.service JobTimeoutSec=15min After=network.target JobTimeoutAction=poweroff-force 76 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 76 2/19/15 9:23 AM You can follow the chain of dependencies if you look at what BASICTARGET REQUIRES AND WANTS Those are actual unit files in the same /lib/systemd/system/ directory. The Requires and Wants directives above are how systemd defines the dependency chain among the units. The Requires directive DENOTES A HARD REQUIREMENT and Wants denotes an optional REQUIREMENT !LSO KEEP IN MIND THAT Requires and Wants don’t imply order. If the After directive isn’t specified, systemd will start the units in parallel. Timer units are also really interesting. They are unit files that contain a [Timer] section and define how the TimeDateD subsystem of

systemd will activate a future event. In these timer units, you can create two types of timers: one that will activate after a time period based on a variable starting point, such as the systems boot, and another that activates at fixed intervals like a cron job. As a matter of fact, timer units are an alternative to cron jobs. One last thing to mention about systemd unit files is that they provide the means to describe easily what to do when a service crashes. You can do that by using the directives Restart or RestartSec in your unit files. This feature allows systemd to take the role of process supervisor as well. systemd refers to the init dæmon executable itself, namely /lib/systemd/systemd, but it also refers to the set of utilities and programs used to manage the system and services. Chief among these utilities is the systemctl program that’s used to manage services. You can use it to enable, start and disable services, find the status of a given service and also list all

loaded units. For example: Q # systemctl enable sshd Q # systemctl start sshd Q # systemctl stop sshd Q # systemctl status sshd Q # systemctl list-units Some Linux distributions, like 2(%,  AND #ENT/3  PROVIDE A compatibility layer that translates 3YS6 AND 5PSTART COMMANDS INTO systemd commands. If you issue the command service sshd status in CentOS 7, you will get the following output: WWW.LINUXJOURNALCOM / MARCH 2015 / 77 LJ251-March2015.indd 77 2/19/15 9:23 AM FEATURE Initializing and Managing Services in Linux: Past, Present and Future Redirecting to /bin/systemctl status sshd.service sshd.service - OpenSSH server daemon Q Display log since last boot: # journalctl -b Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled) Active: active (running) since Mon 2014-12-08 02:01:53 PST; ´12h ago Q Display errors from last boot: # journalctl -b -p err Process: 915 ExecStartPre=/usr/sbin/sshd-keygen (code=exited, ´status=0/SUCCESS) Main PID: 937 (sshd) CGroup:

/system.slice/sshdservice I urge you to look at the documentation of the different schemes presented here to learn more. .937 /usr/sbin/sshd -D Notice that first line of console output above and how it indicates THAT THE 3YS6 STYLE COMMAND WAS redirected to the systemd-style of command. This allows the user to ease into the systemd way of doing things while still allowing the user to leverage the previous skill set. Another really important program in the systemd toolbox is the journalctl utility. It allows you to view and manage the systemd logging subsystem called journald. systemd’s logfile is a binary file and using journalctl really simplifies the user experience. Here are some interesting examples: Q Display full log: # journalctl --all Q Tail the log: # journalctl -f Q Filter log by executable: # journalctl /lib/systemd/systemd Controversies From my vantage point, the future is not 100% certain when it comes to init schemes for Linux. The clear leader, as ) WRITE THIS IN

LATE  IS SYSTEMD ! lot of distributions are adopting it; the LATEST ONES ARE 2(%,  AND $EBIAN  However, the adoption of systemd has been controversial, and these distributions have received a lot of strong feedback from their respective communities. Of note is the Debian technical committee debate that occurred in the Debian mailing list and a complaint by Linus Torvalds himself in the Linux kernel mailing list. systemd is not just an init scheme. It unifies everything that is related to starting and managing system services into a centralized and monolithic whole: user login, cron jobs, network services, virtual TTY management and so on. The use of shell scripts to control system startup has the benefit of providing flexibility, and a lot of 78 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 78 2/19/15 9:23 AM Where every interaction matters. break down your innovation barriers power your business to its full potential When you’re presented with new

opportunities, you want to focus on turning them into successes, not whether your IT solution can support them. Peer 1 Hosting powers your business with our wholly owned FastFiber NetworkTM, solutions that are secure, scalable, and customized for your business. Unsurpassed performance and reliability help build your business foundation to be rock-solid, ready for high growth, and deliver the fast user experience your customers expect. Want more on cloud? Call: 844.8556655 | gopeer1com/linux | Vew Cloud Webinar: Public and Private Cloud LJ251-March2015.indd 79 | Managed Hosting | Dedicated Hosting | Colocation 2/19/15 9:23 AM FEATURE Initializing and Managing Services in Linux: Past, Present and Future members of the community want to be able to choose their favorite init scheme. This has spawned some forks of systemd and even a faction of the Linux community that is for completely boycotting systemd. Check out the site http://boycottsystemd.org influenced user choice

in this space over time. I hope this article will foster further discussion, and your feedback is highly encouraged. Q Jonas Gorauskas is technically a software developer by trade but also a generalist with background in operations. In the past he has been one or more of the following: programmer, technical Conclusion The userspace initialization and management of Linux systems has had a rich and diverse history. I hope that this article has given you a new perspective for how we got to where we are today with systemd becoming the new standard. I have covered all the pros and cons of the different schemes and how those factors have support analyst, technical writer, systems designer, database administrator, amateur cook and professional curmudgeon. Jonas is currently working at Intuit in Reno, Nevada, as part of the Application Operations Engineering team helping them with operations, deployment, DevOps or anything else they can think of. Send comments or feedback via

http://www.linuxjournalcom/contact or to ljeditor@linuxjournal.com Resources The source code of various Linux distributions, including: Q Debian 7 and 8 Q CentOS 6.5 and 7 Q Slackware 14 Q Fedora 20 Q Ubuntu 12.4 and 144 Q Arch Linux The Web site of Lennart Poettering: http://0pointer.net/blog The systemd Documentation: http://freedesktop.org/wiki/Software/systemd Upstart Documentation: http://upstart.ubuntucom/cookbook 80 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 80 2/19/15 9:23 AM NEW! Linux Journal eBook Series GEEK GUIDES Slow Down to Speed Up: Continuous Quality Assurance in a DevOps Environment FREE Down loa NOW d ! By Bill Childers DevOps is one of the newest and largest movements in Information Technology in the past few years. The name DevOps is a portmanteau of “Development” and “Operations” and is meant to denote a fusion of these two functions in a company. Whether or not your business actually does combine the two functions, the lessons

and tools learned from the DevOps movement and attitude can be applied throughout the entire Information Technology space. This eBook focuses on one of the key attributes of the DevOps movement: Quality Assurance. At any point, you should be able to release your product, code or configurationso long as you continue keeping your deliverables in a deployable state. This is done by “slowing down” to include a Quality Assurance step at each point in your workflow. The sooner you catch an error or trouble condition and fix it, the faster you can get back on track. This will lower the amount of rework required and keep your team’s momentum going in a forward direction, enabling your group to move on to new projects and challenges. Build a Private Cloud for Less Than $10,000! By Mike Diehl This eBook presents a compelling argument as to why you should consider re-architecting your enterprise toward a private cloud. It outlines some of the design considerations that you need to be aware

of before implementing your own private cloud, and it describes using the DevCloud installer in order to install OpenStack on an Ubuntu 14 server. Finally, this eBook will familiarize you with the features and day-to-day operations of an OpenStack-based private cloud architecture, all for less than $10K! DOWNLOAD NOW AT: http://linuxjournal.com/geekguides LJ251-March2015.indd 81 2/19/15 9:23 AM FEATURE Infinite BusyBox with systemd Infinite BusyBox with systemd Lightweight virtual containers with PID 1. Charles Fisher 82 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 82 2/19/15 9:23 AM I n this article, I demonstrate a method to build one Linux system within another using the latest utilities within the systemd suite of management tools. The guest OS container design focuses upon BusyBox and Dropbear for the userspace system utilities, but I also work through methods for running more general application software so the containers are actually useful. This

tutorial was developed on Oracle Linux 7, and it likely will run unchanged on its common brethren (Red Hat, CentOS, Scientific Linux), and from here forward, I refer to this PLATFORM SIMPLY AS 6 3LIGHT CHANGES may be necessary on other systemd PLATFORMS SUCH AS 353% $EBIAN OR 5BUNTU  /RACLES 6 RUNS ONLY ON THE X? PLATFORM SO THATS THIS ARTICLES primary focus. Required Utilities Red Hat saw fit to remove the long-included BusyBox binary from ITS 6 DISTRIBUTION BUT THIS EASILY is remedied by downloading the latest binary directly from the project’s Web site. Since the /home filesystem gets a large amount of space by default when installing 6 LETS PUT IT THERE FOR NOW 2UN the commands below as root until indicated otherwise: cd /home wget http://busybox.net/downloads/binaries/latest/busybox-x86 64 You also can get a binary copy of the Dropbear SSH server and client from this location: wget http://landley.net/aboriginal/downloads/ ´binaries/extras/dropbearmulti-x86 64

For this article, I used the following versions: Q BusyBox v1.211 Q $ROPBEAR 33( MULTI PURPOSE V These are static binaries that do not link against shared objectsnothing ELSE IS REQUIRED TO RUN THEM AND THEY are ideal for building a new UNIX-ish ENVIRONMENT QUICKLY Build a chroot The chroot system call and the associated shell utility allow an arbitrary subdirectory somewhere on the system to be declared as the root for all child processes. The commands below populate the “chroot jail”, then lock you in. Note that the call to chroot needs your CHANGE TO THE 3(%,, ENVIRONMENT WWW.LINUXJOURNALCOM / MARCH 2015 / 83 LJ251-March2015.indd 83 2/19/15 9:23 AM FEATURE Infinite BusyBox with systemd BusyBox changes its behavior depending upon how it is calledit bundles a whole system of utility programs into one convenient package. variable below, as you don’t have bash inside the jail (and it’s likely the DEFAULT VALUE OF 3(%,,  export SHELL=/bin/sh mkdir /home/nifty

mkdir /home/nifty/bin cd /home/nifty/bin cp /home/busybox-x86 64 /home/dropbearmulti-x86 64 . chmod 755 busybox-x86 64 dropbearmulti-x86 64 ./busybox-x86 64 --list | awk {print "ln -s ´busybox-x86 64 " $0} | sh chroot /home/nifty export PATH=/bin ls -l ###(try some commands) exit Take some time to explore your shell environment after you launch your chroot above before you exit. Notice that you have a /bin directory, which is populated by soft links that resolve to the BusyBox binary. BusyBox changes its behavior depending upon how it is calledit bundles a whole system of utility programs into one convenient package. Try a few additional UNIX commands that you may know. Some that work are vi , uname , uptime and (of course) the shell that you are working inside. Commands that don’t work include ps , top and netstat . 4HEY FAIL BECAUSE THEY REQUIRE THE /proc directory (which is dynamically provided by the Linux kernel)it has not been mounted within the jail. Note that few

native utilities will run in the chroot without moving many dependent libraries (objects). You might try copying bash or gawk into the jail, but you won’t be able to run them (yet). In this regard, BusyBox is ideal, as it depends upon nothing. Build a Minimal UNIX System and Launch It The systemd suite includes the eponymous program that runs as PID 1 on Linux. Among many other utilities, it also includes the nspawn program that is used to 84 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 84 2/19/15 9:23 AM launch containers. Containers that are created by nspawn fix most of the problems with chroot jails. They provide /proc, /dev, /run and otherwise EQUIP THE CHILD ENVIRONMENT WITH A more capable runtime. Next, you are going to configure a getty to run on the console of the container that you can use to log in. Being sure that you have exited your chroot from the previous step, run the following commands as root: mkdir /home/nifty/etc mkdir /home/nifty/root echo

NAME="nifty busybox container" > ´/home/nifty/etc/os-release cd /home/nifty that appear in the child container also appear on the host system, but different PIDs will be assigned between the parent and child. Note that you’ll also receive the message: “The kernel auditing subsystem is known to be incompatible with containers. Please make sure to turn off auditing with AUDIT ON THE KERNEL COMMAND line before using systemd-nspawn. Sleeping for 5s.” The audit settings don’t seem to impact the BusyBox container login, but you can adjust your kernel command line in your grub configuration (at least to silence the warning and stop the delay). ln -s bin sbin ln -s bin usr/bin echo root::0:0:root:/root:/bin/sh > ´/home/nifty/etc/passwd echo console::respawn:/bin/getty 38400 /dev/console > ´/home/nifty/etc/inittab tar cf - /usr/share/zoneinfo | (cd /home/nifty; tar xvpf -) systemd-nspawn -bD /home/nifty After you have executed the nspawn above, you will be

presented with a “nifty login” prompt. Log in as root (there is no passwordyet), and try a few more commands. You immediately will notice that ps and top work, and there is now a /proc. You also will notice that the processes Running Dropbear SSH in Your Container It’s best if you configure a nonroot user of your system and forbid network root logins. The reasoning will become clear when I address container security. Run all of these commands as root within the container: cd /bin ln -s dropbearmulti-x86 64 dropbear ln -s dropbearmulti-x86 64 ssh ln -s dropbearmulti-x86 64 scp ln -s dropbearmulti-x86 64 dropbearkey ln -s dropbearmulti-x86 64 dropbearconvert WWW.LINUXJOURNALCOM / MARCH 2015 / 85 LJ251-March2015.indd 85 2/19/15 9:23 AM FEATURE Infinite BusyBox with systemd Above, you have established the names that you need to call Dropbear, both the main client and server, and the sundry key generation and management utilities. You then generate the host keys that will be

used by this container, placing them in a new directory /home/nifty/ etc/dropbear (as viewed by the host): mkdir /etc/dropbear dropbearkey -t rsa -f /etc/dropbear/dropbear rsa host key dropbearkey -t dss -f /etc/dropbear/dropbear dss host key dropbearkey -t ecdsa -f /etc/dropbear/dropbear ecdsa host key 6ARIOUS DIRECTORIES ARE THEN CREATED that you will need shortly: echo root:::::::: > /etc/shadow chmod 600 /etc/shadow echo root:x:0: > /etc/group passwd -a x root Note that the BusyBox passwd call used here generated an MD5 hashthere is a $1$ prefix in the second field of /etc/shadow for root. Additional hashing algorithms are available from this version of the passwd utility (the options -a s will generate a $5$ SHA256 hash, and -a sha512 will generate a $6$ hash). However, Dropbear seems to be able to work only with $1$ hashes for now. Finally, add a new user to the system, and then halt the container: mkdir -p /var/log/lastlog mkdir /home adduser -h /home/luser -D luser

mkdir /var/run passwd -a x luser mkdir /tmp mkdir /var/tmp halt chmod 01777 /tmp /var/tmp You then create the inittab, which will launch syslogd and Dropbear once at startup (in addition to the existing getty that is respawned whenever it dies): echo ::sysinit:/bin/syslogd >> /etc/inittab echo ::sysinit:/bin/dropbear -w -p 2200 >> /etc/inittab Next, you add a shadow file and create a password for root: You should see container shutdown messages that are similar to a system halt. When you next start your container, it will listen on socket 2200 for connections. If you want remote hosts to be able to connect to your container from anywhere on the network, run this command as root on the host to open a firewall port: iptables -I INPUT -p tcp --dport 2200 --syn -j ACCEPT 86 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 86 2/19/15 9:23 AM The port will be open only until you reboot. If you’d like the open port to persist across reboots, use the

firewall-config command from within the X Window System (set the port on the second tab in the GUI). In any case, run the container with the previous nspawn syntax, then try to connect from another shell within the parent host OS with the following:  BIT LIBRARIES AVAILABLE WITH AN argument to nspawn that establishes a bind mount: ssh -l luser -p 2200 localhost 9OU THEN WILL FIND THAT MANY  BIT binaries that you copy in from the host will run (running /bin/gawk -V RETURNS h.5 !WK vˆAN ENTIRE Oracle 12c instance is confirmed to run this way). The read-only library bind mount also has the benefit of receiving security patches immediately when they appear on the host. There is a significant security problem with this, however. The root user in the container has the power to You should be able to log in to the luser account under a BusyBox shell. Executing Programs with Runtime Dependencies If you copy various system programs from /bin or /usr/bin into your container, you

immediately will notice that they don’t work. They are missing shared objects that they need to run. If you had previously copied the gawk binary in from the host: cp /bin/gawk /home/nifty/bin/ you would find that attempts to execute it fail with “gawk: not found” errors (on the host, there usually will be explicit complaints about missing shared objects, which are not seen in the container). You easily can make most of the systemd-nspawn -bD /home/nifty --bind-ro=/usr/lib64 Then, from within the container, run: cd / ln -s usr/lib64 lib64 mount -o remount,rw /usr/lib64 and, thus, gain write access to your host library directories. In general, you cannot give root to a container user that you don’t know and trust among other problems, these mounts can be abused. You also might be tempted to mount the /usr/lib directory in the same manner. The difficulty you will find is that the systemd binary will be found under that directory WWW.LINUXJOURNALCOM / MARCH 2015 / 87

LJ251-March2015.indd 87 2/19/15 9:23 AM FEATURE Infinite BusyBox with systemd tree, and nspawn will try to execute it in preference to BusyBox init. %NABLING  BIT RUNTIME SUPPORT likely will involve more directory and mounting gymnastics than was REQUIRED FOR USRLIB And now, I’m going off on a tangent. systemd Service Files You will need to call on the host PID 1 (systemd) directly to launch your container in an automated manner, potentially at boot. To do this, you need to create a service file. Because there is a dearth of clear discussion on moving inittab and service functions into systemd, I’ll cover all the basic uses before creating a service file for the container. Start by configuring a telnet server. The telnet protocol is not secure, as it transmits passwords in clear text. Don’t practice these examples on a production server or with sensitive information or accounts. Classical telnetd is launched by the inetd superserver, both of which are implemented by

BusyBox. Let’s configure inetd for telnet on port 12323. Run the following as root on the host: echo 12323 stream tcp nowait root ´/home/nifty/bin/telnetd telnetd -i -l /home/nifty/bin/login >> /etc/inetd.conf After the configuring above, if you manually launch the inetd contained in BusyBox, you will be able to telnet to port 12323. Note THAT THE 6 PLATFORM DOES NOT include a telnet client by default, so you either can install it with yum or use the BusyBox client (which the example below will do). Unless you open up port 12323 on your firewall, you will have to telnet to localhost. Make sure any inetd that you started is shut down before proceeding to create an inetd service file below: echo [Unit] Description=busybox inetd #After=network-online.target Wants=network-online.target [Service] #ExecStartPre= #ExecStopPost= #Environment=GZIP=-9 #OPTION 1 ExecStart=/home/nifty/bin/inetd -f Type=simple KillMode=process #OPTION 2 #ExecStart=/home/nifty/bin/inetd #Type=forking 88

/ MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 88 2/19/15 9:23 AM #Restart=always S #User=root Kernel 3.100-12393el7x86 64 on an x86 64 #Group=root localhost.localdomain login: jdoe Password: [Install] WantedBy=multi-user.target > ´/etc/systemd/system/inetd.service Checking the status again, you see information about the connection and the session activity: systemctl start inetd.service [root@localhost ~]# systemctl status inetd.service After starting the inet service above, you can check the status of the dæmon: inetd.service - busybox inetd Loaded: loaded (/etc/systemd/system/inetd.service; disabled) Active: active (running) since Sun 2014-11-16 12:34:04 CST; ´7min ago [root@localhost ~]# systemctl status inetd.service inetd.service - busybox inetd Main PID: 3927 (inetd) CGroup: /system.slice/inetdservice Loaded: loaded (/etc/systemd/system/inetd.service; disabled) ´3927 /home/nifty/bin/inetd -f Active: active (running) since Sun 2014-11-16

12:21:29 CST; ´4076 telnetd -i -l /home/nifty/bin/login ´28s ago ´4077 -bash Main PID: 3375 (inetd) CGroup: /system.slice/inetdservice ´3375 /home/nifty/bin/inetd -f Nov 16 12:21:29 localhost.localdomain systemd[1]: Started ´busybox inetd. Try opening a telnet session from a different console: /home/nifty/bin/telnet localhost 12323 You should be presented with a login prompt: Entering character mode Escape character is ^]. You can learn more about systemd service files with the man 5 systemd.service command There is an important point to make hereyou have started inetd with the “-f Run in foreground” option. This is not how inetd normally is started this option is commonly used for debugging activity. However, if you were starting inetd with a classical inittab entry, -f would be useful in conjunction with “respawn”. Without -f , inetd immediately will fork into the background; attempting to WWW.LINUXJOURNALCOM / MARCH 2015 / 89 LJ251-March2015.indd 89 2/19/15

9:23 AM FEATURE Infinite BusyBox with systemd respawn forking dæmons will launch them repeatedly. With -f , you can configure init to relaunch inetd should it die. Another important point is stopping the service. With a foreground dæmon and the KillMode=process setting in the service file, the child telnetd services are not killed when the service is stopped. This is not the normal, default behavior for a systemd service, where all the children will be killed. To see this mass kill behavior, comment out the OPTION 1 settings in the service file (/etc/systemd/ system/inetd.service), and enable the default settings in OPTION 2 . Then execute: systemctl stop inetd.service systemctl daemon-reload systemctl start inetd.service Launch another telnet session, then stop the service. When you do, your telnet sessions will all be cut with “Connection closed by foreign host.” In short, the default behavior of systemd is to kill all the children of a service when a parent dies. The

KillMode=process setting can be used with the forking version of inetd, but the “-f Run in foreground” in the first option is more specific and, thus, safer. You can learn more about the KillMode option with the man 5 systemd.kill command Note also that the systemctl status output included the word “disabled”. This indicates that the service will not be started at boot. Pass the enable keyword to systemctl for the service to set it to launch at boot (the disable keyword will undo this). Make some note of the commented options above. You may set environment variables for your service (here suggesting a COMPRESSION QUALITY SPECIFY A NON ROOT user/group and commands to be executed before the service starts or after it is halted. These capabilities are beyond the direct features offered by the classical inittab. Of course, systemd is capable of spawning telnet servers directly, allowing you to dispense with inetd altogether. Run the following as root on the host to configure

systemd for BusyBox telnetd: systemctl stop inetd.service echo [Unit] Description=mytelnet [Socket] 90 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 90 2/19/15 9:23 AM ListenStream=12323 Accept=yes [Install] to the telnet server indicates that systemd should not pay attention to any stats return codes from the process. WantedBy=sockets.target > ´/etc/systemd/system/mytelnet.socket echo [Unit] Description=mytelnet Q In the client telnet sessions, the command cat /proc/self/ cgroup will return detailed connection information for the IP addresses involved. [Service] ExecStart=-/home/nifty/bin/telnetd telnetd -i -l ´/home/nifty/bin/login StandardInput=socket > ´/etc/systemd/system/mytelnet@.service systemctl start mytelnet.socket At this point, I have returned from my long-winded tangent, so now let’s build a service file for the container. Run the following as root on the host: echo [Unit] Description=nifty container Some notes about inetd-style

services: [Service] Q The socket is started, rather than the service, when inetd services are launched. Similarly, they are enabled to set them to launch at boot. Q The @ character in the service file indicates this is an “instantiated” service. They are used when a number of similar services are launched with a single service file (getty being the prime example they also work well for Oracle database instances). Q The - prefix above in the path ExecStart=/usr/bin/systemd-nspawn -bD /home/nifty KillMode=process > /etc/systemd/system/nifty.service Be sure that you have shut down any other instances of the nifty container. You optionally can disable the console getty by commenting/removing the first line of /home/nifty/etc/inittab. Then use PID 1 to launch your container directly: systemctl start nifty.service If you check the status of the service, you will see the same level of WWW.LINUXJOURNALCOM / MARCH 2015 / 91 LJ251-March2015.indd 91 2/19/15 9:23 AM FEATURE

Infinite BusyBox with systemd information that you previously saw on the console: [root@localhost ~]# size /home/busybox-x86 64 text data bss dec hex 942326 29772 19440 991538 f2132 filename /home/busybox-x86 64 [root@localhost ~]# systemctl status nifty.service nifty.service - nifty container Loaded: loaded (/etc/systemd/system/nifty.service; static) Active: active (running) since Sun 2014-11-16 14:06:21 CST; ´31s ago Main PID: 5881 (systemd-nspawn) CGroup: /system.slice/niftyservice ´5881 /usr/bin/systemd-nspawn -bD /home/nifty If you want to conserve the memory used by BusyBox, one way would be to create a common /cbin that you attach to all containers as a read-only bind mount (as you did previously WITH LIB AND RESET ALL THE LINKS IN /bin to the new location. The root user could do this: Nov 16 14:06:21 localhost.localdomain systemd[1]: Starting ´nifty container. systemctl stop nifty.service Nov 16 14:06:21 localhost.localdomain systemd[1]: Started ´nifty

container. Nov 16 14:06:26 localhost.localdomain systemd-nspawn[5881]: mkdir /home/cbin mv /home/nifty/bin/busybox-x86 64 /home/cbin ´Spawning namespace container on /home/nifty mv /home/nifty/bin/dropbearmulti-x86 64 /home/cbin ´(console is /dev/pts/4). cd / Nov 16 14:06:26 localhost.localdomain systemd-nspawn[5881]: ´Init process in the container running as PID 5883. ln -s home/cbin cbin cd /home/nifty/bin for x in *; do if [ -h "$x" ]; then rm -f "$x"; fi; done Memory and Disk Consumption BusyBox is a big program, and if you are running several containers that each have their own copy, you will waste both memory and disk space. It is possible to share the “text” segment of the BusyBox memory usage between all running programs, but only if they are running on the same inode, from the same filesystem. The text segment is the read-only, compiled code of a program, and you can see the size like this: /cbin/busybox-x86 64 --list | awk {print "ln

-s ´/cbin/busybox-x86 64 " $0} | sh ln -s /cbin/dropbearmulti-x86 64 dropbear ln -s /cbin/dropbearmulti-x86 64 ssh ln -s /cbin/dropbearmulti-x86 64 scp ln -s /cbin/dropbearmulti-x86 64 dropbearkey ln -s /cbin/dropbearmulti-x86 64 dropbearconvert You also could arrange to bindmount the zoneinfo directory, saving a little more disk space in the container (and giving the container patches for time zone 92 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 92 2/19/15 9:23 AM It might interesting to launch tens, hundreds, or even thousands of containers at once. data in the bargain): cd /home/nifty/usr/share rm -rf zoneinfo Then the service file is modified to bind /cbin and /usr/share/zoneinfo (note the altered syntax for sharing /cbin below, when the paths differ between host and container): echo [Unit] Description=nifty container [Service] ExecStart=/usr/bin/systemd-nspawn -bD /home/nifty --bind-ro=/home/cbin:/cbin --bind-ro=/usr/share/zoneinfo KillMode=process >

/etc/systemd/system/nifty.service systemctl daemon-reload systemctl start nifty.service Now any container using the BusyBox binary from /cbin will share the same inode. All versions of the BusyBox utilities running in those containers will share the same text segment in memory. Infinite BusyBox It might interesting to launch tens, hundreds, or even thousands of containers at once. You could launch the clones by making copies of the /home/nifty directory, then adjusting the systemd service file. To simplify, you will place your new containers in /home/nifty1, /home/nifty2, /home/nifty3 . using integer suffixes on the directories to differentiate them. Please make sure that you have disabled kernel auditing to remove the five-second delay when launching containers. At the very least, press e at the grub menu at boot time, and add the audit=0 to your kernel command line for a one-time boot. I’m going to return to the subject of systemd “instantiated services” that I touched upon

with the telnetd service FILE THAT REPLACED INETD 4HIS TECHNIQUE will allow you to use one service file to launch all of your containers. Such a service has an @ character in the filename that is used to refer to a particular, differentiated instance of a service, and it allows the use of the %i placeholder within the WWW.LINUXJOURNALCOM / MARCH 2015 / 93 LJ251-March2015.indd 93 2/19/15 9:23 AM FEATURE Infinite BusyBox with systemd service file for variable expansion. Run the following on the host as root to place your service file for instantiated containers: make a thousand of them: cd /home for x in $(seq 1 999) do echo [Unit] mkdir "nifty${x}" Description=nifty container # %i (cd nifty; tar cf - .) | (cd "nifty${x}"; tar xpf -) sed "s/2200/$((x+2200))/" < nifty/etc/inittab > [Service] ExecStart=/usr/bin/systemd-nspawn -bD /home/nifty%i ´--bind-ro=/home/cbin:/cbin --bind-ro=/usr/share/zoneinfo ´nifty${x}/etc/inittab systemctl

start nifty@${x}.service done KillMode=process > /etc/systemd/system/nifty@.service The %i above first adjusts the description, then adjusts the launch directory for the nspawn . The content that will replace the %i is specified on the systemctl command line. To test this, make a directory called /home/niftyslick. The service file doesn’t limit you to numeric suffixes. You will adjust the SSH port after the copy. Run this as root on the host: As you can see below, this test launches all containers: $ ssh -l luser -p 3199 localhost The authenticity of host [localhost]:3199 ([::1]:3199) ´cant be established. ECDSA key fingerprint is 07:26:15:75:7d:15:56:d2:ab:9e: ´14:8a:ac:1b:32:8c. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added [localhost]:3199 (ECDSA) ´to the list of known hosts. luser@localhosts password: ~ $ sh --help cd /home BusyBox v1.211 (2013-07-08 11:34:59 CDT) multi-call binary mkdir niftyslick (cd nifty; tar cf - .) | (cd

niftyslick; tar xpf -) sed "s/2200/2100/" < nifty/etc/inittab > niftyslick/etc/inittab systemctl start nifty@slick.service Bearing this pattern in mind, let’s create a script to produce these CONTAINERS IN MASSIVE QUANTITIES ,ETS Usage: sh [-/+OPTIONS] [-/+o OPT]. [-c SCRIPT ´[ARG0 [ARGS]] / FILE [ARGS]] Unix shell interpreter ~ $ cat /proc/self/cgroup 10:hugetlb:/ 9:perf event:/ 94 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 94 2/19/15 9:23 AM 8:blkio:/ ´2882 /bin/dropbear -w -p 2210 7:net cls:/ 6:freezer:/ Nov 18 23:01:21 localhost.localdomain systemd[1]: 5:devices:/ ´Starting Container nifty10. 4:memory:/ Nov 18 23:01:21 localhost.localdomain systemd[1]: 3:cpuacct,cpu:/ ´Started Container nifty10. 2:cpuset:/ 1:name=systemd:/machine.slice/machine-nifty999scope The output of systemctl will list each of your containers: # systemctl . machine-nifty1.scope loaded active running Container nifty1 machine-nifty10.scope loaded

active running Container nifty10 machine-nifty100.scope loaded active running Container nifty100 machine-nifty101.scope loaded active running Container nifty101 machine-nifty102.scope loaded active running Container nifty102 The raw number of containers that you can launch with this approach is more directly impacted by kernel limits than general disk and memory resources. Launching the containers above used no swap on a small system with 2GB of RAM. After you have investigated a few of the containers and their listening ports, the easiest and cleanest way to get all of your containers shut down is likely a reboot. . More detail is available with systemctl status : machine-nifty10.scope - Container nifty10 Loaded: loaded (/run/systemd/system/machine-nifty10.scope; ´static) Drop-In: /run/systemd/system/machine-nifty10.scoped ´90-Description.conf, 90-Sliceconf, ´90-TimeoutStopUSec.conf Active: active (running) since Tue 2014-11-18 23:01:21 CST; ´11min ago CGroup:

/machine.slice/machine-nifty10scope ´2871 init ´2880 /bin/syslogd Container Security A number of concerns are raised with these features: 1) Since BusyBox and Dropbear were not installed with the RPM host package tools, updates to them will have to be loaded manually. It will be important to check from time to time if new versions are available and if any security flaws have been discovered. If it is necessary to load new versions, the binaries should be copied to all containers that are potentially used, which should then be restarted (especially if a security WWW.LINUXJOURNALCOM / MARCH 2015 / 95 LJ251-March2015.indd 95 2/19/15 9:23 AM FEATURE Infinite BusyBox with systemd The crux is that untrusted users cannot have the container root, any more than you would give them full system root. issue is involved). 2) Control of the root user in the container cannot be passed to an individual that you do not trust. For a particular example, if the LIBCBINZONEINFO BIND MOUNTS

above are used, the container root user can issue the command: mount -o remount,rw /usr/lib64 at which point the container root will have full write privileges on your  BIT LIBRARIES CONTAINER BIN OR ZONEINFO The systemd-nspawn man page goes even further, with the warning: Note that even though these security precautions are taken systemd-nspawn is not suitable for secure container setups. Many of the security features may be circumvented and are hence primarily useful to avoid accidental changes to the host system from the container. The intended use of this program is debugging and testing as well as building of packages, distributions and software involved with boot and systems management. The crux is that untrusted users cannot have the container root, any more than you would give them full system root. The container root will have the CAP SYS ADMIN privilege, which allows full control of the system. If you want to isolate non-root users further, the container environment

does limit non-root users’ visibility into host activities, as they cannot see the full process table. 3) Note that the BusyBox su and passwd utilities above do not work when installed in the manner outlined here. They lack the appropriate filesystem permissions. To fix this, chmod u+s busybox-x86 64 could be executed, but this is also distasteful from a security perspective. Removing the links and copying the BusyBox binary to su and passwd before applying the setuid privilege 96 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 96 2/19/15 9:23 AM might be better, but only slightly. It would be best if su was unavailable and another mechanism was found for password changes.  4HE -w argument to the Dropbear SSH server above prevents root logins from the network. It is somewhat distasteful, from a security perspective, to relax this limitation. The net effect is that root is locked out of active use in the container when -w is forced, and su/passwd do not have setuid.

If it is at all possible to live with such an arrangement for your container, try to do so, as the security is much improved. systemd Controversy There is a high degree of hostility toward systemd from users of Linux. This hostility is divided into two main complaints: Q The classic inittab from UNIX 3YSTEM 6 SHOULD NOT BE CHANGED because it is well understood. 6 INIT IS NOT ABLE TO NSPAWN AND HAS far less control over processes running on a system. The features delivered by systemd surely justify the inconvenience of change in many situations. Toward the second point, much thought was placed into the adoption of the architecture of systemd by skilled designers from diverse organizations. Those most critical of the new environment should acknowledge the technical success of systemd as it is adopted by the majority of the Linux community. In any case, the next decade will see popular Linux server distributions EQUIPPED WITH SYSTEMD AND COMPETENT administrators will not have the option

of ignoring it. It is unfortunate that the introduction of systemd did not include more guidance for the user community, but the new features are compelling and should not be overlooked. Q Charles Fisher has an electrical engineering degree from the University of Iowa and works as a systems and database Q Increasing features are bundled into systemd that bring dangerous complexity to a critical system process. Toward the first point, nostalgia for legacy systems is not always misguided, but it cannot be allowed to hinder progress unreasonably. A classic System administrator for a Fortune 500 mining and manufacturing corporation. He has previously published both journal articles and technical manuals on Linux for UnixWorld and other McGraw-Hill publications. Send comments or feedback via http://www.linuxjournalcom/contact or to ljeditor@linuxjournal.com WWW.LINUXJOURNALCOM / MARCH 2015 / 97 LJ251-March2015.indd 97 2/19/15 9:23 AM KNOWLEDGE HUB WEBCASTS Learn the 5 Critical

Success Factors to Accelerate IT Service Delivery in a Cloud-Enabled Data Center Todays organizations face an unparalleled rate of change. Cloud-enabled data centers are increasingly seen as a way to accelerate IT service delivery and increase utilization of resources while reducing operating expenses. Building a cloud starts with virtualizing your IT environment, but an end-to-end cloud orchestration solution is key to optimizing the cloud to drive real productivity gains. > http://lnxjr.nl/IBM5factors Modernizing SAP Environments with Minimum Riska Path to Big Data Sponsor: SAP | Topic: Big Data )S THE DATA EXPLOSION IN TODAYS WORLD A LIABILITY OR A COMPETITIVE ADVANTAGE FOR YOUR BUSINESS %XPLOITING MASSIVE AMOUNTS of data to make sound business decisions is a business imperative for success and a high priority for many firms. With rapid advances in x86 processing power and storage, enterprise application and database workloads are increasingly being moved from UNIX to Linux

as part of IT modernization efforts. Modernizing application environments has numerous TCO and ROI benefits but the transformation needs to be managed carefully and performed with minimal downtime. Join this webinar to HEAR FROM TOP )$# ANALYST 2ICHARD 6ILLARS ABOUT THE PATH YOU CAN START TAKING NOW TO ENABLE YOUR ORGANIZATION TO GET THE benefits of turning data into actionable insights with exciting x86 technology. > http://lnxjr.nl/modsap WHITE PAPERS White Paper: JBoss Enterprise Application Platform for OpenShift Enterprise Sponsor: DLT Solutions 2ED (ATSš *"OSS %NTERPRISE !PPLICATION 0LATFORM FOR /PEN3HIFT %NTERPRISE OFFERING PROVIDES )4 ORGANIZATIONS WITH A SIMPLE AND STRAIGHTFORWARD WAY TO DEPLOY AND MANAGE *AVA APPLICATIONS 4HIS OPTIONAL /PEN3HIFT %NTERPRISE COMPONENT FURTHER EXTENDS THE DEVELOPER AND MANAGEABILITY BENEFITS INHERENT IN *"OSS %NTERPRISE !PPLICATION 0LATFORM FOR ON PREMISE CLOUD ENVIRONMENTS 5NLIKE OTHER MULTI PRODUCT OFFERINGS THIS IS NOT A

BUNDLING OF TWO SEPARATE PRODUCTS *"OSS %NTERPRISE -IDDLEWARE HAS BEEN HOSTED ON THE /PEN3HIFT PUBLIC OFFERING FOR MORE THAN  MONTHS !ND MANY CAPABILITIES AND FEATURES OF *"OSS %NTERPRISE Application Platform 6 and JBoss Developer Studio 5 (which is also included in this offering) are based upon that experience. This real-world understanding of how application servers operate and function in cloud environments is now available in this SINGLE ON PREMISE OFFERING *"OSS %NTERPRISE !PPLICATION 0LATFORM FOR /PEN3HIFT %NTERPRISE FOR ENTERPRISES LOOKING FOR CLOUD benefits within their own datacenters. > http://lnxjr.nl/jbossapp 98 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 98 2/19/15 9:23 AM KNOWLEDGE HUB WHITE PAPERS Linux Management with Red Hat Satellite: Measuring Business Impact and ROI Sponsor: Red Hat | Topic: Linux Management Linux has become a key foundation for supporting todays rapidly growing IT environments. Linux is being used to deploy

business applications and databases, trading on its reputation as a low-cost operating environment For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows IN IMPORTANCE IN TERMS OF VALUE TO THE BUSINESS MANAGING ,INUX ENVIRONMENTS TO HIGH STANDARDS OF SERVICE QUALITY ˆ AVAILABILITY SECURITY AND PERFORMANCE ˆ BECOMES AN ESSENTIAL REQUIREMENT FOR BUSINESS SUCCESS > http://lnxjr.nl/RHS-ROI Standardized Operating Environments for IT Efficiency Sponsor: Red Hat 4HE 2ED (ATš 3TANDARD /PERATING %NVIRONMENT 3/% HELPS YOU DEFINE DEPLOY AND MAINTAIN 2ED (AT %NTERPRISE ,INUXš AND THIRD PARTY APPLICATIONS AS AN 3/% 4HE 3/% IS FULLY ALIGNED WITH YOUR REQUIREMENTS AS AN EFFECTIVE AND MANAGED process, and fully integrated with your IT environment and processes. Benefits of an SOE: 3/% IS A

SPECIFICATION FOR A TESTED STANDARD SELECTION OF COMPUTER HARDWARE SOFTWARE AND THEIR CONFIGURATION FOR USE ON COMPUTERS WITHIN AN ORGANIZATION 4HE MODULAR NATURE OF THE 2ED (AT 3/% LETS YOU SELECT THE MOST APPROPRIATE solutions to address your business IT needs. SOE leads to: s $RAMATICALLY REDUCED DEPLOYMENT TIME s 3OFTWARE DEPLOYED AND CONFIGURED IN A STANDARDIZED MANNER s 3IMPLIFIED MAINTENANCE DUE TO STANDARDIZATION s )NCREASED STABILITY AND REDUCED SUPPORT AND MANAGEMENT COSTS s 4HERE ARE MANY BENEFITS TO HAVING AN 3/% WITHIN LARGER ENVIRONMENTS SUCH AS s ,ESS TOTAL COST OF OWNERSHIP 4#/ FOR THE )4 ENVIRONMENT s -ORE EFFECTIVE SUPPORT s &ASTER DEPLOYMENT TIMES s 3TANDARDIZATION > http://lnxjr.nl/RH-SOE WWW.LINUXJOURNALCOM / MARCH 2015 / 99 LJ251-March2015.indd 99 2/19/15 9:23 AM EOF Resurrecting the Armadillo DOC SEARLS Fifteen years after giving the world a pile of hopefully helpful memes, Cluetrain rides again. 1999 was a crazy year for business on

the Internet, and for Linux. It was when Red Hat went public, with a record VALUATION AND 6! ,INUX FOLLOWED with a bigger one. Both were cases in point of the dot-com boom, a speculative bubble inflated by huge expectations of what the Internet would mean for business. In April of that year, Chris Locke, Rick Levine, David Weinberger and I put up a Web site called The Cluetrain Manifesto (http://cluetrain.com), attempting to make clear that the Internet was for everybody and everything, and not just for companies looking to exit into wealth on Wall Street. Our bulls-eye was marketing, which spoke about users in terms that were barely human. “We are not seats or eyeballs or end users or consumers”, Cluetrain said. “We are human beings and our reach exceeds your grasp. Deal with it” We addressed Cluetrain to “People OF %ARTHv BUILT IT AROUND  THESES (because that worked for Martin Luther) and called it a “manifesto” (because that worked for Karl Marx). The

“Cluetrain” name came from an OLD 3ILICON 6ALLEY PUT DOWN h4HE clue train stopped there four times a day and they never took delivery.” )T WAS A HIT 6OLUNTEERS TRANSLATED it into 13 languages. The Wall Street Journal covered it on the front page of its Marketplace section. Book offers came in. We accepted one and wrote the book version of The Cluetrain Manifesto that summer. It came out in January 2000 and was a business bestseller, even though it could also be read for free on-line at the Cluetrain Web site. It went on 100 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 100 2/19/15 9:23 AM EOF New Clues was tuned for our timeone in which Internet usage has been migrating from wired to cellular connections, from the Web to apps, and from the Net’s wide open spaces to the closed and proprietary walled gardens of Facebook, Twitter, Apple, Google and other feudal lords. to be published in nine languages, and still sells at a nice clip, 15 years later. Same goes

for a 10th anniversary edition that came out in 2010. Today the word cluetrain, which didn’t exist before 1999, appears in thousands of books and is tweeted many times per day. One-liners from its list of theses“Markets are conversations”, “Hyperlinks SUBVERT HIERARCHYvˆARE QUOTED endlessly. Not bad for an project that had no promotion, no budget, no conferences, no bumper stickers, no t-shirts and no interest in becoming a business or an institution. It was just a bunch of ideas people could put to use. The biggest irony of Cluetrain’s success is that most books that cite it are marketing books, and most tweets about it seem to be by marketing people. Is marketing better because of it? To some degree, I suppose. But I don’t care Nor do I care that Cluetrain is often credited with having something to do with social media. What I care about is that Cluetrain hasn’t yet succeeded at its main mission: to make clear that the Internet is something more than the pipes we get

it from, the “content” we find there, and the companies and governments that would have us think they run the thing. 3O ON !UGUST   WHEN somebody pointed me to yet another Cluetrain story that failed to grok what it was really about, I wrote this to the other three guys: “I feel an urge to publish something that says ’The Cluetrain Manifesto was not about clearing the way for social media.’” David Weinberger wrote back, “Anyone ready for a new manifesto?” A few minutes later, he shared the first draft of one: a collection of theses, similar to the WWW.LINUXJOURNALCOM / MARCH 2015 / 101 LJ251-March2015.indd 101 2/19/15 9:23 AM EOF original in style and length. Chris Locke followed with a wordless image of a woman gleefully shooting thumbs-up images out of a machine gun. The rest of the back-and-forth was between David and me. (Chris and Rick stayed busy with other things.) The result was New Clues, which went up on January 8, 2015, at the original Cluetrain

site: http://cluetrain.com/newclues New Clues was tuned for our timeone in which Internet usage has been migrating from wired to cellular connections, from the Web to apps, and from the Net’s wide open spaces to the closed and proprietary walled gardens of Facebook, Twitter, Apple, Google and other feudal lords. “An organby-organ body snatch of the Internet is already well underway”, it says in the preamble. “Make no mistake: with a stroke of a pen, a covert handshake, or by allowing memes to drown out the cries of the afflicted we can lose the Internet we love.” One hundred and twenty one numbered clues follow, under thematic subheads: Q The Internet is not content. Q The Net is not a medium. Q How did we let conversation get weaponized anyway? Q Marketing still makes it harder to talk. Q Kumbiyah sounds surprisingly good in an echo chamber. Q The Gitmo of the Net. Q Gravity’s great until it sucks us all into a black hole. Q Privacy in an age of spies. Q Privacy in an

age of weasels. Q A pocket full of homilies. Q Being together: the cause of and solution to every problem. We wanted to make New Clues, and every piece of it, as useful, mixable and remixable as possible. So: Q %VERY SUBHEAD AND EVERY CLUE HAS A Q The Internet is us, connected. Q The Internet is nothing and has no purpose. link of its own. Q The text is released to the public domain with a Creative Commons 102 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 102 2/19/15 9:23 AM EOF We wanted to make New Clues, and every piece of it, as useful, mixable and remixable as possible. :ERO  5NIVERSAL ##  Public Domain Dedication. In other words, no copyright at all. When asked for permission to republish New Clues, David replies, “You don’t have our permission. Go ahead!” the great Steven Levy): https://medium.com/backchannel/ internet-under-fire-gets-newmanifests-207a922b459e. Q A version by Kevin Marks that accepts fragmentations and webmentions. Q The

whole thing is on GitHub, with machine-readable versions (JSON, XML and OPML, so far). The GitHub folks also have offered to set up a way for us to maintain a single data file (YAML) that will automatically create all the other versions we need. The results so far (I’m writing this a week after it went up) include: Q A listicle that Dave W iner hacked together on his own software, which he improved in the course of posting it. (Some people like the listicle version better than the one-page text one. Try it out: http://scripting.com) Q An artful posting on Backchannel at Medium (by invitation from Q A site by John Johnston that randomly generates one clue per click: http://johnjohnston.info/ oddsandends/givemeaclue. Q Thousands of tweets and re-tweets (hashtags: #cluetrain #newclues). Q 6OLUNTEER TRANSLATIONS INTO ERMAN Italian, Italian (yes, there are two different ones), Catalan and French (see Resources). Q Lots of great blog posts, such as this one by JP Rangaswami:

http://confusedofcalcutta.com/ 2015/01/11/new-clues-calling-oneveryone-to-be-dutiful-individuals. Q A Gillmor Gang devoted to to the subject, with David and WWW.LINUXJOURNALCOM / MARCH 2015 / 103 LJ251-March2015.indd 103 2/19/15 9:23 AM EOF Now the question is, Will it work? Or will it be, like so much else that gets published on the Web, snow on the water? myself: http://techcrunch.com/2015/ 01/10/gillmor-gang-kind-of-clue. Q A discussion page on Facebook: https://www.facebookcom/ login.php?next=https%3A%2F%2F www.facebookcom%2Fgroups% 2Fnewclues%2F. Our only common design element between Cluetrain and New Clues is an armadillo. On Cluetrain’s index page is the image shown in Figure 1 of a flattened armadillo in the middle of a road, painted over with yellow divider lines. The provenance of the photo is unknown to us. Chris Locke found it somewhere, and nobody has ever stepped forward to claim it. The one for New Clues is shown in Figure 2. It was posted by e res on Flickr

and made useful by a Creative Commons Attribution 2.0 Generic (CC BY 2.0) license (The shot was taken at Alki Beach in Seattle, the town where Linux Journal was born.) For New Clues, we cropped the shot and made it black and white. For his Figure 1. Armadillo from Cluetrain’s Index Page listicle version, Dave Winer kept the color but darkened it. Among the few criticisms of New Clues is that it’s “not so disruptive” as Cluetrain was. For the last few YEARS 3ILICON 6ALLEY HAS BEEN SO GAGA over disruption that it even has a conference named after it. The term 104 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 104 2/19/15 9:23 AM comes from Clayton Christensen’s work on disruptive innovation, which is defined as “a process by which a product or service takes root initially in simple applications at the bottom of a market and then relentlessly moves up market, eventually displacing established competitors”. This can apply to ideas as well as technologies.

Cases in point: free software and open source. Both of which, of course, informed Cluetrain and New Clues. .OW THE QUESTION IS Will it work? Or will it be, like so much else that gets published on the Web, snow on the water? Can’t say, so soon after it’s published. But the two publishing dates, a decade and a half apart, came at very different times on the Web, and we did our best to leverage both. In 1999, the Web was a collection of almost physical places. You put up or built Web sites on domains with locations that people visited and browsed. Search engines might take days or weeks to index a page. But then, after blogs came along, with syndication through RSS, what my son Allen in 2003 described as “the Live Web” began to emerge. Technorati and other search engines for live stuff appeared. My October 2005 column in Linux Journal was titled “The World Live Web”. In it I said the Live Web was “about time and people, rather than Advertiser Index Thank you as always for

supporting our advertisers by buying their products! ADVERTISER URL PAGE # Drupalize.me http://drupalize.me %MBEDDED ,INUX Conference HTTPEVENTSLINUXFOUNDATIONORG events/embedded-linux-conference  %MPEROR,INUX HTTPWWWEMPERORLINUXCOM  HPC Wallstreet http://www.flaggmgmtcom/linux 31 Libre Planet 2015 https://libreplanet.org/2015/ 21 LinuxFest Northwest http://linuxfestnorthwest.org/2015 65 Netgate http://www.netgatecom /g2EILLY 3OFTWARE Architecture Conference HTTPOREILLY#BB+)  Peer 1 Hosting http://go.peer1com/linux 79 Silicon Mechanics http://www.siliconmechanicscom 32%CON HTTPSWWWUSENIXORGCONFERENCE  srecon15 6AULT HTTPEVENTSLINUXFOUNDATIONORG events/vault 108 7 3  ATTENTION ADVERTISERS The Linux Journal brand’s following has grown to a monthly readership nearly one million strong. Encompassing the magazine, Web site, newsletters and much more, Linux Journal offers the ideal content environment to help

you reach your marketing objectives. For more information, please visit http://www.linuxjournalcom/advertising WWW.LINUXJOURNALCOM / MARCH 2015 / 105 LJ251-March2015.indd 105 2/19/15 9:23 AM EOF Figure 2. New Clues Armadillo sites and content”. This was a year before Twitter and Facebook took off, and search engines’ time-to-index was reduced nearly to zero. Sites today are geysers of content, and the Live Web is a giant short-attention-span theater. What shows there is hyper-social and temporary in the extreme, made more for sharing than for saving. This is why New Clues is a collection of stuff for sharing, presented atop an old chunk of bedrock, which is what Cluetrain has become. If we had only put it on Medium, or on Facebook, it would have come and gone in a week. As buzz, that’s pretty much what it did. But as durable food for re-thinking what the Net is, and how we might lose it, maybe it will have lasting effects. Sure hope so Q Doc Searls is Senior Editor of

Linux Journal. He is also a fellow with the Berkman Center for Internet and Society at Harvard University and the Center for Information Technology and Society at UC Santa Barbara. Send comments or feedback via http://www.linuxjournalcom/contact or to ljeditor@linuxjournal.com 106 / MARCH 2015 / WWW.LINUXJOURNALCOM LJ251-March2015.indd 106 2/19/15 9:23 AM EOF Resources Dot-Com Bubble: http://en.wikipediaorg/wiki/Dot-com bubble Chris Locke: http://rageboy.com Rick Levine: https://twitter.com/ricklevine David Weinberger: http://www.hyperorgcom/blogger The Cluetrain Manifesto: http://cluetrain.com The entire original text of The Cluetrain Manifesto: http://cluetrain.com/book “What The Cluetrain Manifesto Teaches Us On Social Media.11 Years Later”: http://visionarymarketing.com/en/blog/2010/02/what-the-cluetrain-manifesto-teaches-uson-social-media-11-years-later The Cluetrain Legacy and Social Media: http://www.chrisgcom/cluetrain-social-media Invasion of the Body Snatchers:

http://en.wikipediaorg/wiki/Invasion of the Body Snatchers CC0 1.0 Universal (CC0 10) Public Domain Dedication: http://creativecommons.org/publicdomain/zero/10 e. res on Flickr: https://wwwflickrcom/photos/iamtheloop Creative Commons Attribution 2.0 Generic License: https://creativecommonsorg/licenses/by/20 Backchannel: New Clues: https://medium.com/backchannel/internet-under-fire-gets-new-manifests-207a922b459e Steven Levy: http://www.stevenlevycom John Johnston: http://johnjohnston.info German Translation: http://conceptbakery.de/blog/2015/01/11/new-clues-deutscheuebersetzung-die-neuen-thesen-von-den-verfassern-des-cluetrain-manifest Italian Translation 1: https://medium.com/bee-free-the-social-bee/cluetrain-15-anni-dopo-9d6b4def4d57 Italian Translation 2: https://medium.com/@nuovetesi/nuove-tesi-4a1def360351 Catalan Translation: https://ca.wikisourceorg/wiki/New clues “New Clues: Calling on everyone to be Dutiful Individuals” by JP Rangaswami:

http://confusedofcalcutta.com/2015/01/11/new-clues-calling-on-everyone-to-be-dutiful-individuals Gillmor Gang: Kind of Clue: http://techcrunch.com/2015/01/10/gillmor-gang-kind-of-clue Facebook Discussion Page: https://www.facebookcom/login php?next=https%3A%2F%2Fwww.facebookcom%2Fgroups%2Fnewclues%2F Disrupt Conference: http://techcrunch.com/event-type/disrupt Clayton Christensen: http://www.claytonchristensencom Disruptive Innovation: http://www.claytonchristensencom/key-concepts “Snow on the Water”: http://blogs.lawharvardedu/doc/2014/08/03/snow-on-the-water “The World Live Web” by Doc Searls in the October 2005 issue of LJ: http://www.linuxjournalcom/article/8549 WWW.LINUXJOURNALCOM / MARCH 2015 / 107 LJ251-March2015.indd 107 2/19/15 9:23 AM Instant Access to Premium Online Drupal Training Instant access to hundreds of hours of Drupal training with new videos added every week! Learn from industry experts with real world H[SHULHQFHEXLOGLQJKLJKSURȴOHVLWHV Learn on

the go wherever you are with apps for iOS, Android & Roku We also offer group accounts. Give your whole team access at a discounted rate! Learn about our latest video releases and RIIHUVȴUVWEIROORZLQJXVRQ)DFHERRNDQG 7ZLWWHU #GUXSDOL]HPH  Go to http://drupalize.me and get Drupalized today! LJ251-March2015.indd 108 2/19/15 9:23 AM