Recently in linux Category

One of the tasks I have is to build some separation between the production environment and test/integration environments. As we approach that Very Big Deadline I've talked about, this is a pretty significant task. Dev processes that we've been using for a really long time now need to change in order to provide the separation we need, and that's some tricky stuff.

One of the ways I plan on accomplishing this is through the use of Environments in Puppet. This looks like a great way to allow automation while delivering different code to different environments. Nifty keen stuff.

But it requires that I re-engineer our current Puppet environment, which is a single monolithic all-or-nothing repo to accommodate this. This is fairly straight-forward, as suggested by the Puppet Docs:

If the values of any settings in puppet.conf reference the $environment variable (like modulepath = $confdir/environments/$environment/modules:$confdir/modules, for example), the agent's environment will be interpolated into them.

So if I need to deliver a different set of files to production, such as a different authorized_keys file, I can drop that into $confdir/environments/production/modules and all will be swell.

So I tried to set that up. I made the environment directory, changed the puppet.conf file to reflect that, made an empty module directory for the module that'd get the changed files (but not put the files in there yet), and ran a client against it.

It didn't work so well.

The error I was getting on the puppetmaster was err: Could not find class custommod for clunod-wk0130.sub.example.local at /etc/puppet/manifests/site.pp:84 on node clunod-wk0130.sub.example.local

I checked eight ways from Sunday for spelling mistakes, bracket errors, and other such syntax problems but could not find out why it was having trouble locating the custommod directory. I posted on ServerFault for some added help, but didn't get much beyond, "Huh, it SHOULD work that way," which wasn't that helpful.

I decided that I needed to debug where it was searching for modules since it manifestly wasn't seeing the one staring it in the face. For this I used strace, which produced an immense logfile. Grepping out the 'stat' calls as it checked the import statements, I noticed something distinctly odd.

stat("/etc/puppet/modules/base/manifests/init.pp"
stat("/etc/puppet/modules/curl/manifests/init.pp"
stat("/etc/puppet/environments/production/modules/custommod/manifests/init.pp"

It was attempting to read in the init.pp file for the custommod, even though there wasn't a file there. And what's more, it wasn't then re-trying under /etc/puppet/modules/custommod/manifests/init.pp. Clearly, the modulepath statement does not work like the $PATH variable in bash and every other shell I've used. In those, if it fails to find a command in (for example) /usr/local/bin, it'll then try /usr/bin, and then ~/bin until it finds the file.

The modulepath statement in puppet.conf is focused on modules, not individual files.

This is a false cognate. By having that empty module in the production environment, I was in effect telling Puppet that the Production environment doesn't have that module. Also, my grand plan to provide a different authorized_keys file in the Production environment requires me to have a complete module for the Production environment directory, not just the one or two files I want changed.

Good to know.

Today I ended up banging my head on a particularly truculent wall. A certain clustering solution just wasn't doing the deed. No matter how I cut, sliced, sawed or hacked it wouldn't come to life. I wanted my two headed beast! Most vexing.

Then I hit that point where I said unto myself (I hope), "!#^#^ it, I'm reformatting these two and starting from scratch."

So I did.

It worked first time.

^)(&*#q)*%

Ok, I'll take it.

Mind, all the cutting and slicing and sawing and hacking was with me knowing full well that iterative changes like that can stack unknowingly, and I was backing out the changes when I determined that they weren't working. Clearly, my back-out methods weren't clean enough. Okay, food for thought; dirty uninstallers are hardly an unusual thing.

Of course I kicked myself for not trying that earlier, but during the course of aforementioned reanimatory activity I did learn a fair amount about this particular clustering technology. Not a complete loss. I do wish I had those 4 hours back, since we're at a part of the product deployment schedule where hours count.

This weekend I put together (mostly as it turned out) a new fileserver for home. The box that's currently pulling that duty is 9 years old and still working just fine.

However, 9 years old. I'm happily surprised that 40GB hard-drive is still working. Only 58,661 power-on hours, and all of 3 reallocated sectors. But still, 9 years old. There comes a time when hardware just needs to be retired before it goes bad in spectacular ways.

The system that is replacing this stalwart performer cost under $400 with shipping, which contrasts starkly with the sticker price on the old unit which was around $1400 back in the day. This system is based on the AMD E350 CPU. Underpowered, but this is a file-server and that ancient P4 is still keeping up. I have a Drobo that I'll be moving to this unit, which is where the bulk of the data I want to serve is being kept anyway.

As for the title of the post, this E350 mobo has a UEFI firmware on it. That's when things got complicated.

I was just catching up on email when I ran into this interesting tidbit: 

Up to now, openSUSE users had the choice of using openJDK (GPL with classpath exceptions) or Sun/Oracle's Java. The Sun/Oracle Java was licensed under the "Distributor's License for Java (DLJ)", which allowed Linux distributors to package and redistribute Sun/Oracle Java. Recently, Oracle announced that openJDK 7 is the new official reference implementation for Java SE7. They no longer see the need for the DLJ licensed Java implementation and so have retired that license.

openSUSE chooses to proceed with distributing the GPL licensed official reference implementation, openJDK. We will no longer distribute newer versions or updates of the now proprietary licensed Sun/Oracle JDK. Existing installations of the Sun/Oracle JDK are still licensed under the now retired DLJ.

openSUSE users who wish to continue using the Sun/Oracle JDK (including new versions thereof and updates) should now download directly from http://www.oracle.com/java. For now we keep the current sun-java packages (under the DLJ license) in the Java:sun:Factory project and will not update them anymore.

I suggest to document in the openSUSE wiki how to install the Sun/Oracle JDK version from Oracle under openSUSE.

Which is to say, Oracle is killing the license that allows OpenSUSE to provide Sun/Oracle Java as a part of the repo.

As a user of java apps in the line of my work, this deeply annoys me. What few management java apps work well on Linux (it's always a surprise when something does), seem to work best with Oracle Java not OpenJDK..

It probably won't be just OpenSUSE affected by this.

This is a question from ServerFault that was there and then was no longer there because it's rampant flame-bait and got mod-hammered. But sometimes flame bait can make for good blog-posts, so here it is. Unattributed since the source no longer exists and I don't want to embarrass the asker.

As someone who has a good amount of experience with basic server setup exclusively on Linux, I'm wondering why anybody would want to use Windows.

I'm not asking this to make it into some snide comment, I just don't see any advantages.

The big things I think I would miss are:

• SSH access. As far as I know, the only real way to remotely access a Windows service is via RDP or VNC or something similar, which is a lot more work if all I want to do is restart a service.
• Open source software. From my experience, almost all open source server software is made for Linux. While some, like Apache, can also be run on Windows, a lot of the times it feels like it was added as an afterthought.
• Easy configuration. I've never used Windows tools, but I love being able to apt-get install libapache2-mod-whatever. While package systems aren't technically part of Linux, most popular distributions use yum or aptitude or some packaging system which makes it a lot easier to handle updates.

Again, I've not used Windows extensively as a server, so please forgive me if some of these points are inaccurate.

A valid question. We had a thread much like this one on the LOPSA mailing list a while ago. And really, to a Linux admin, Windows looks like an expensive, opaque, and above all annoying way of doing what Linux can do in its sleep. This view is very couched in the observer's biases.

The consensus of the web this year is that if you want to do large scale web-application infrastructures, Linux is where it is at in spades. During my job hunt there were exceedingly few job-postings for Linux admins that mentioned something other than Web or DB duties. Web, DB, load-balancing, routing, orchestration, caching layers, it's all there and very well documented.

So why WOULD you use use Windows?

The number one reason I know of...

Because the application you're using requires it.

At WWU we had quite a number of off-the-shelf products require a Windows server because they were .NET applications. FLOSS versions may exist, but that's not what our users wanted. They wanted this piece of software that they picked out and is kinda standard in their industry, not some half baked open source project out of some other University.

Or for my current employer, a number of the key processing tasks we need to do are most accurately accomplished on Windows. The open source versions of these software packages get close enough, but part of what distinguishes us from our competitors is that we get closer then that.

The number two reason...

Because that's what you know.

This was why WWU was running Blackboard on Windows servers, even though it's a Tomcat application at the core. I'm pretty sure the reason for this is because what came before Blackboard was also running on Windows and our Windows admin inherited the new environment, not that the Linux admin said "Not it!" faster than the Windows admin. I know that admin found Linux confusingly opaque and convoluted.

The number three reason...

Because you don't have time/skill to maintain it yourself, and/or you're willing to pay someone else to do it for you.

If that application comes in a box, wrapped in an installation wizard, and comes complete with phone-home abilities to pull updates, notify the vendor (and later you) of problems, a lot of the effort in keeping that application going has now been outsourced to the vendor. Few FLOSS-stack products can do that, they need some skilled time to keep 'em up. To an organization looking to fire-and-forget, this kind of software is really attractive.

---

Now on to some of the asker's specific concerns regarding remote access, scalability, and software installs. Below the fold.

HP has been transitioning away from the cciss Linux kernel-driver for a while now, but there hasn't been much information about what it all means. Just on the name alone the module needed a rename (one possible acronym of cciss: Compaq Command Interface for SCSI-3 Support), and it is a driver that has been in the Linux ecosystem a really long time (at least in the 2.2 kernel era). A lot has changed in the kernel.

HP has finally released a PDF describing the whole cciss vs. hpsa thing.

Read it here: http://h20000.www2.hp.com/bc/docs/support/SupportManual/c02677069/c02677069.pdf

The key differences:

• HPSA is a SCSI driver, not a block-driver like CCISS
• This means that the devices are moving from /dev/cciss to /dev
• Device mode numbers will change
• New controllers will increment kernel names, so a second controller will be /dev/sda, not /dev/sdb, so use udev names (partition ID, disk-ID, that kind of thing) to avoid pain.
  
• For newer kernels (2.6.36+) cciss and hpsa can load at the same time if the system contains hardware that needs those drivers.
  

In the beginning, there was Slackware. And lo, was it good. The kernels did compile, and there were software packages aplenty. For the rest, there was 'make'.

And then there came RedHat with support contract in hand, promising both stability and accountability. Business distrust of 'free' was assuaged by accountability, and the quiet uptake began. To better manage their profit-generating systems, RedHat begat RPM.

And so did professional geekdom learn to use RPM, for with it came pay-checks. And with the engines of commerce running RedHat, the major software vendors did release their software packaged with RPM. RPM being a robust solution, other Linux projects did adopt RPM as their own package manager.

But quiet as a mouse Canonical did release a distribution that users found easy and simple. It just worked, they cried. Their DEB package manager did match RPM in both flexibility and scalability. Professional geekdom did install it at home, or ignored it to their peril.

Time passed, and more private users were using Ubuntu than any other distribution. Professional geekdom did stop paying RedHat support contracts for large installs, preferring instead to use the free CentOS. 'Patches-only!' being all that professional geekdom deemed needed for most scalable web infrastructures.

Quietly beneath senior professional geekdom a whole generation of new geeks had come of age under Ubuntu and DEB and not Fedora and RPM. Canonical did release a Long Term Support version of their distribution, designed specifically to provide code stability for several years and did also provide the option of a support contract which did supply accountability.

The frothy churn of startups saw Ubuntu, not CentOS in use far more often than even some years before. So much so, that professional geekdom, in their inter-career travels, ran into it more and more. What worked for RPM did not work for DEB, though they did solve the same problem. As with all startup churn, some few did make it big with their Ubuntu infrastructure supporting them.

And so did DEB come of age in the ranks of professional geekdom.

Signal boosting this one:



Survey expires on the 29th, so you probably have less than 24 hours from when I hit submit to fill it out.

What is it?

The Ada Initiative (http://adainitiative.org/) is a newly-formed organisation which aims to support and promote women in open technology and culture. We've just launched our first annual census -- a broad survey of open technology and culture participants -- to find out more about what projects and communities people are involved in, and how they feel about women's inclusion and representation in the field.

Not just for women! Men-folk can fill it out as well, as it's also asking about perceptions of openness in various places. The fields I tend to hang around in tend to be, ahem, dinosaur pits¹ , which I duly noted. Others, though, are more diverse². So please do look into this survey.

---

1: A seminar I attended on high performance computing had over 100 people in the room, only five of which were female, and three of whom were attached to the vendor doing the presentation. The ratio is about the same for storage-focused talks.
2:The general System Administrator field does have more women these days, this is a good thing.

On the OpenSuse factory list the question has been asked: Why do you still use Windows?

This generated a very long thread with the usual back-n-forth, but I figured I'd blog about why I'm still using Windows privately; work is another matter. In broad strokes there are three reasons:

1. There are some applications I want to use that don't exist on Linux, and also don't like to work in Wine.
2. There are still some devices that only work right with Windows software to unlock their features.
3. Gaming on Windows looks and works much better.
   

Neither of these are things the FOSS community can do much about. #2 can be helped by reverse-engineering the software on a per-device basis to make them work right, and there is quite a bit of guerrilla work on that right now. #1 has been a problem since version 0.01 and will be with us for quite some time, though there are some commercial options available that help fix it. #3 is a subset of #1, but is probably the most visible deficit to a pure-Linux environment.

Right now I'm solving the first two through virtualization, and the third by having a dedicated gaming machine. Like a console, but Windows, and a lot more expensive. My at-home Windows use is pretty small these days, an exception.

Work, as I said, is another story. While I was at WWU, I was a Domain Administrator with all the duties that entails, and that job is much easier if you're running native. My next job is more unknown, but I do know they have a Windows environment for one task, but I won't know the full extent of it until I get there (possibly late next week).

The historic 'rdesktop' product (located here) hasn't been updated in a while, and critically lacks support for a major new feature Microsoft introduced in 2008. Network-level authentication. This enhancement requires a login to even connect the RDP session. The rdesktop client hasn't been able to support this, so for my servers that I wish to RDP to I have to remember to turn that off.

Happily, the FreeRDP fork of rdesktop now has support for NLA. It's in the GIT repo, not the stable branch, but it is there. The next stable version should have support for NLA.

Yay!