What is Shoes Federales?

federales64I need to step away from the coding for a little bit, take a deep breath, collect my thoughts and [insert favorite metaphor here]. Since I’m approaching a release milestone, I’m going to state my original goals and my progress and deviance from them. Along the way, I’ll do some self serving cheer leading and toss out hints of future activities. Because I do things that way.

The problem I wanted to solve was that Shoes (Red Shoes, aka 3.0 and 3.1) has been languishing from neglect while Shoes 4 is being developed. So I set out to do a maintenance release. Truth be told, even before I decided that, I thought Shoes would be a neat thing to have on my new Raspberry Pi and other Pi folks might like Shoes and I would gain some cross compiling skills by building Shoes on my big Ubuntu box. My original goals a few months ago:

  1. Ruby 1.9.3 support (2.x.y too?)
  2. Gtk3 support
  3. Linux only enhancements. No changes for MsWin or OSX
  4. Fix gem support. Allow system wide gems to be used or improve Shoes ability to manage its own gems.
  5. Packaging – create .deb for shoes, shoes+app+gems
  6. Clean up rakefiles
  7. Improve cross compilation – specifically to Raspberry pi

Before I discuss each goal, please note that I do not have a goal of competing with the Shoes 4. I have not added any new features to the Shoes Language or layout. No new widgets. I did fix a bug or two and I did add a couple of Constants to Shoes but I also deprecated some features – primarily user level packaging of Shoes apps. That has been seriously un-maintained and it continues to be un-maintained. So, how have I done?

1. Ruby 1.9.3.
This took a lot longer than I expected. You could always compile Shoes with Ruby 1.9.3 but you might also get a screenful of complaints out of Ruby about constants and/or gems. I don’t like warnings unless I know enough to ignore them. It turns out there were two or three problems. Ruby 1.9.3 hardcodes the location of ‘rubygems.rb’ For example, mine was deep in /home/ccoupe/.rvm/ and on your machine it might be /home/yourname/rvm or /usr/lib/ruby1.9. Well, that explains why no one can build a binary Shoes on linux and expect it to work on someone else’s machine. It turns out that Ruby can be compiled with ‘–enable-load-relative’ (H/T to Michal Papis – Thanks). Load relative does look for rubygems.rb relative to where the running ruby is in the file system. For more fun, you should know that when it finds rubygems.rb one of the first things is to look for rbconfig.rb. Ruby has the benefit of keeping track of where it loads things from and there can be multiple places to load from. Ruby will do it’s best to look in all of them until it gives up. It also means that if ruby loads rubygems.rb from /home/ccoupe/.rvm… and then shoes.rb loads rubygems or rbconfig from ./shoes/ruby/lib it is a different rbconfig (as far as Ruby is concerned they came from different load paths so they are different files. That all happens before before the first bit of Shoe’s ruby code starts. Way before cache.rb attempts to clean up the mess. Lesson: compile your Ruby with –enable-load-relative if you can.

The next problem is that when Ruby goes looking for rbconfig.rb (or rubygems.rb) at start up, load relative or not, it is hardcoded to look for lib/ruby/1.9.1/{arch}.rbconfig.rb and it turns out Shoes stores its copy of Ruby in ruby/lib/{arch}rbconfig.rb. Puzzling things can happen in that case including the fact that many Shoes things work just fine, but not all Shoes things. You might never know what went wrong if you ignore those console error messages. The choice I made was to get Shoes’ copy of Ruby to look like lib/ruby/1.9.1 (note the switch between lib/ruby and ruby/lib – that took me too long to find). While I was poking around in app.c, world.c and friends I removed any chance of using Ruby 1.8.7 – I’m a deprecating machine, I ya.

I also found a way to jailbreak Gems from the Shoes sandbox (more on that later) and a way to compile Shoes on Linux that doesn’t require copies of Ruby, jpeg, [un]gif and so on. For obvious reasons, that Shoes binary will never have a chance of working on anyone else’s machine. I call it Loose Shoes. Some could reasonably argue it should be called Tight Shoes because it’s tightly tied to the users OS and configuration. It’s also smaller in size without a dangling copy of Ruby to drag around. At the time I was thinking of the Shoe Sandbox and binary distributions where everything has to be tightly controlled for things to work. So Loose Shoes turns out to be a build from source with whatever you have for a Ruby and Tight Shoes is a Shoes built for binary distribution and all the dependencies are tightly controlled. Like I said, there’s a good argument for the opposite meaning. But, having decided, I moved on.

2. Gtk3 support
Since all recent Linux distributions include Gtk3 and not Gkt2 (they also don’t include the development headers for either, but ignore that pesky fact in pursuit of truth, beauty and the American Way). Clearly, Gkt3 is the future (like it or not). So I added Gkt3 support to Shoes (gtk.c, version.h, common.h and later, world.c). I also left the Gkt2 code in so I can compile Shoes either way just by setting a Constant in the Rakefiles. That turned out to be a good idea. I still have a couple of show stopping Gtk3 bugs to fix before that version of Shoes hits the interwebs. I also noticed that Gkt3 and Gtk2 also claim to have Windows versions and Gtk3 runs on OSX/Quartz so no X11 on OSX required. Very interesting possibilities if that works. More on that later. So, I don’t have Gkt3 support fulling working because I was chasing shiny things that appeal to me more.

3. Linux only enhancements. No changes for MsWin or OSX
Almost all of my work was done on Linux. That’s almost the same thing, right? The OSX part was probably mostly true.

4. Fix gem support. Allow system wide gems to be used or improve Shoes ability to manage its own gems.

Early on, I decided that I didn’t want Shoes to include built in gems and extensions. Nobody, including me, has ever wanted to maintain a copy of Hpricot and Sqlite. They aren’t included or built in Federales (the source code is still there like an appendix). Nobody loves the binject extension or wants to maintain it. Bloopsaphone and portaudio? Gone (aka not built). Minitar – not maintained – probably still works for shy. My thought was to replace that with Zip – I think there is a ruby api/gem that would work and there’s also an api in rubygems for tar files that I haven’t looked into. I kept chipmunk because I like it and ftsearch because the manual likes it. So two extensions, no gems included.

For Loose Shoes (compiled from source for your Linux box and pre-existing Ruby), Shoes will use Gems you’ve installed in the system or with rvm. It will also use gems in ~/.shoes/gems+ so if you have some older gems in gem+ expect some confusion and error messages.

For Tight Shoes (Shoes intended to be distributed as binaries), in cache.rb, it checks for a file in ~/.shoes/federales named getoutofjail.card. If it sees that file, then it adds ENV[‘GEM_HOME’] to the Shoes/Ruby load path and the end users RVM gems are available for Shoes scripts. I have done nothing to improve or upgrade Shoes own gem handling code. As far as I’m concerned the only thing I intend to do with Shoes.setup is to delete it one of these days. It probably doesn’t work in Federales. I don’t know. I don’t intend to find out.

On the to-do list is to write a separate program in Shoes to manage gems (any gems or gemset or bundle). I’m not likely to get to that for quite a while. It would be a nice task for someone who wants to write a moderately difficult Shoes script or two. they could even use the Shoes.setup code that I threatened to delete as as a starting point.

5. Packaging – create .deb for shoes, shoes+app+gems
Not a whole lot progress to report on this. I did get nsis working to build Windows exe’s. Does that count for anything? Probably not. The fpm gem could do most of the heavy lifting here but I’ve got a lot of questions and haven’t bothered to look for answers.

6. Clean up rakefiles
Lets just agree to disagree about whether I improved the rakefiles. I deleted the bundler/cucumber stuff. The OSX folks can add it back in if they care to work on Shoes 3.2 and if it actually does something useful for them. If you just want to compile Shoes in Linux, it’s not needed and it gets in the way – a negative contribution. I commented out the check with github to get a version number. Ever try to compile Shoes when the internet connection is down? Very annoying while it waits for github to timeout. Yeah, it needs to be put back in. Perhaps when a package is created would be a good time to get a new version number. Also, it always returns 1540 for me no matter what I commit to github so it’s kind of useless as a version number.

If you get the feeling that I don’t particularly like the rake build system, you would be correct. But, it mostly works, it exists, and it can be abused for good purposes which is what I did when I got around to cross compiling and building binary distributions (Tight Shoes). The default is to build Shoes for whatever platform ruby is running (Tight Shoes) That will build a static libshoes.a and include that in the shoes program. Minimal dynamic linking for Loose Shoes, just against the dependent libs on your system. No funny copying of unneeded libraries. There was one very small change to chipmunk.rb needed for that to work. No need for the launching script that sets LD_LIBRARY_PATH either. No confusion with Loose Shoes about where things live. Just ask ldd. I added an install task which copies Loose Shoes to ~/.shoes/federales and produces a .desktop file that might work on your Linux Menu system when copied to the proper directory. since Ruby doesn’t move (or get copied), I can do this. It’s giggalishly small. But you can’t share it because it won’t work over there.

I also complicated the crap out things by adding some linux:setup: namespace tasks so you can cross compile (Loose Shoes) for Linux-x86_64, Linux-i686, Raspberry, and several variations of MinGW. More on that below. I know every one loves abstraction, but sometimes it’s very annoying. This is one of those times. It’s just freaking compile and link of C programs and some file copies and file system links. I still question whether I should have moved to Cmake but I didn’t. There is a set of files (env.rb and tasks.rb) for each Tight Shoes target (which gcc calls ‘host’ so beware of the shifting terminology). env.rb is mostly setting up constants that tasks.rb uses. There is commonality but in this case, don’t get elegant because every cross compile setup is going to be different and until you’ve done it enough times on enough platforms you many lack the knowledge of what has to be done and over engineer an abstraction that is worse than none at all.

That said – I did clean up the rakefiles and then I polluted them with a collection of cross compiling targets that work for me. They won’t work for you unless you create a setup like mine and there is no commandment that you have to set things up my way and in the Cecil way I left plenty of hints and unused code that can be used to do it differently.

So, I don’t like the Rakefiles or my changes. But it works and I don’t think I’ve done anything that would prevent Windows folks from building Shoes with their MinGW/Msys/Rubyinstaller setup nor have I intentionally destroyed anything OSX folks need should either of the OSX or MinGW folks attempt to build Shoes3.2

7. Improve cross compilation – specifically to Raspberry pi
Yes, I did that and for a whole lot more than the raspberry. It deserves a separate and longer post to describe how it works for me. Tight Shoes — binary distributions are cross compiled. Even if you are on a 64 bit linux and want to build a 64 bit linux binary of Shoes, you need to set up a cross compile arrangement. Why? Re-read point #1 of this post. You can’t use your installed Ruby and expect it to work anywhere but your own system. You have to build a separate Ruby and distribute that (and any .so’s that Ruby wants like libyaml, openssl and so on) as well as the regular shoes dependencies of libjpeg, libgif/libungif (or both). You just have to do it. If you want to argue that point, try it your way and then we can compare notes about theory and facts.

Enough grumpiness.
I’m delighted that I can build Shoes for Windows using the native widgets, Gtk2 or Gtk3. I can build 32 bit linux and 64 bit linux binaries that runs on recent Suse, Fedora, and Debian without requiring the end user to install anything. I can do the same for the Raspberry Pi. I have to fix the Shoes Gtk3 bugs before that’s worth much more than a brag. Still, it’s good progress. There are Windows threading problems and most of that solution should be done in Ruby so that could be a problem I have to solve. I’m also hopeful that Gtk3 on OSX would allow Shoes 3.2 (or 3.3) without anyone having to learn much about Objective-C or Cocoa. I’m not likely to do the the OSX work, but I’m hopeful someone will step up. Red shoes is not dead — It’s in the recovery room. Upgraded from Critical to stable.

Oh, one more thing. Or two things. I created a new icon and named my fork Federales because I’m just different and ornery. Federales is a rift on Policeman so there is some continuity in the naming. The icon background is the Mexican flag for obvious pun reasons.

Leave a Reply

Your email address will not be published. Required fields are marked *