[aklug] Re: Node Alchemy

From: Everett Haimes <everett@powernode.net>
Date: Wed Feb 06 2013 - 15:36:52 AKST

When I started the rewrite of the server portion of the project it was somewhere in the ~2.3 version of rails back in 2009. I checked my git repo and I started version controlling on October 14, 2010 at Rails version 3.0.1 after an extensive (and slightly daunting) conversion.

Since the upgrade to 3.0 I haven't hit any major migration issues and things have been extremely stable. I've been able to continually update the rails version without issue and am now running 3.2.11. I've been extremely happy with it and development has been very smooth. Right now I'd recommend Rails to anyone interesting in designing a webapp with complex object-relational models.

I'm currently investigating the upgrade path to Rails 4.0 but will not upgrade until they have a few patch level releases out.

As for versioning, with RVM and Bundler, dependencies are extremely easy to deal with and in a few specific cases, I've had to lock a version down on a gem here and there. An example is the latest JQuery update. Some of my Javascript is not compatible with the latest version so I've locked the version of the jquery-rails gem until I'm able to make it compatible again.

Thanks,
Everett Haimes

----- Original Message -----
From: "Arthur Corliss" <acorliss@nevaeh-linux.org>
To: "Everett Haimes" <everett@powernode.net>
Cc: aklug@lib.uaa.alaska.edu
Sent: Wednesday, February 6, 2013 3:10:27 PM
Subject: Re: [aklug] Node Alchemy

On Wed, 6 Feb 2013, Everett Haimes wrote:

> Hello,
>
> I received a message from a user concerned about information going out about Node Alchemy. I'd like to introduce myself and hopefully start a dialog about the service and see if anyone would like to get involved.
>
> My name is Everett Haimes, currently employed at a Wide Area Network Administrator III for the North Slope Borough School District in Barrow, AK. I've been working with Linux since 1995 and have been working on this project since 2005 when I wrote the init portion of this software. Initially this consisted of a directory structure containing SquashFS modules and used SSH to list, download, and attach them to the system root. Since that time I've been working on a management interface, a poller, manager, proxy and store and use authenticated web requests with simple CSV API for nodes, and a JSON API the other components, all through authenticated HTTPS requests.
>
> The goal of this project is to simplify the management of similarly configured node instances based on standardized service modules, and node and instance specific dependency modules.
>
> Right now I've got a fully functional system using the following components:
>
> Server (Ruby on Rails 3.2): Management interface, used to define nodes based on templates, subscribe to modules, manage versions, schedule operations, etc.
>
> Manager: Sidekiq based worker queue performs backend work such as committing modules, checking status, creating/destroying cloud instances, and executing commands on node instances.
>
> Poller: Ruby daemon that pulls node info from server and schedules jobs for the manager.
>
> Proxy: Server API compatible Sinatra server that acts as a module store, locally caches modules with the store, and relays requests to the server.
>
> Store: Sidekiq based worker queue that concurrently downloads requested modules and provides them to the proxy.
>
> Modules are defined with what I call specifications which are nothing more than a list of files used by rsync to construct the modules. There is a primary spec and dependency spec. When a node is subscribed to a module, a module subscription is created and based on that subscription, dependency modules can be created. The dependency modules inherit their specs from the subscription module's dependency spec and can be a config module which is shared between all node instances (physical and cloud) as well as instance modules which are bound to a specific instance. Instance modules can be committed directly from it's instance with a single command and would be used for things such as hostname, IP configuration, etc. Subscription and config modules are managed/modified on a primary cloud instance and committed with the manager by scheduling a commit within the web interface.
>
> Within the web interface, physical instances can be defined and a LiveCD specific for that instance can be scheduled for creation with the click of a button. The manager creates an ISO with all the most recent modules and uploads it to the server where a download link appears. If you boot to that ISO, it will check for updates during boot and update all newer modules but use all current ones from the CD.
>
> I've built in features such as copy-to-ram where it will copy the modules to RAM before mounting which speeds things up considerable, and with the ability to netboot, you can spin up physical instances such as servers without hard drives.
>
> Volatile data storage can be added with "mount points" that can be defined in the web interface, including mount point dependencies allowing you to mount a volume, then --bind mount directories to arbitrary locations in the system, in the correct order.
>
> As for the init system, I've build a custom initramfs with a full network stack including DHCP/DNS support. On boot, the init attempts to connect to the parent (or proxy) and downloads a list of current modules. It then downloads/verifies the modules and adds them to the union root and continues init at usual.
>
> This is a very new service and I currently don't have a library of modules other than a base Ubuntu 12.04 server module and generic credentials module to allow logins. Ideally there will need to be a library of subscription modules for things like server software to make this useful.
>
> If anyone would like to get involved or learn more, please email me and I'd be happy to elaborate.
>
> Thanks for your time,
>
> Everett Haimes
> Node Alchemy LLC
> ---------
> To unsubscribe, send email to <aklug-request@aklug.org>
> with 'unsubscribe' in the message body.

It's good to hear from the developer of this project. I was surprised to
see the Ruby on Rails bit, which leads me to a bit of a side question:
those of you doing production work on Ruby, how stable a platform is that
with updates?

Back in the day I had others write Ruby code that I had to support on my
distro. Started out okay, but fell apart rapidly with even minor updates to
the Ruby engine or various gems. I swore off Ruby at that point becuase I
didn't want to have to test freaking everything to see if some small,
supposedly innocuous patch would break half of the Ruby apps.

Have things gotten any better? Or do you just rely on your distro
maintainer to backport fixes?

         --Arthur Corliss
           Live Free or Die
---------
To unsubscribe, send email to <aklug-request@aklug.org>
with 'unsubscribe' in the message body.
Received on Wed Feb 6 15:37:06 2013

This archive was generated by hypermail 2.1.8 : Wed Feb 06 2013 - 15:37:06 AKST