Federales Part 2: Shoes dependencies and cross compiling

This post is long on words and short on details. If I listed every command required, this would a multi-chapter and woefully out of date before you could read to the end.

As I said in the previous post, What is Shoes Federales, you can’t depend on Ruby being compiled and installed with the proper options that would allow you build a Binary Distribution of Shoes. And even if it was proper, thats only one architecture and their are many architectures. You could setup Virtual Machines with OracleBox or VMWare and install two or three Linux and Windows OS’s and then install all the development tools needed to build Shoes and then if you have some way to sync the Shoes source code (git clones, nfs, rsync, samba). You log in to each VM, build and package Shoes and all is good. You could do that. I started to do that. There are quite a few obstacles on that path but once you’ve learned that the default ruby you install on the xx VM can’t work for anybody else w/o rebuilding it properly you’ll look for a better way. Don’t get me wrong – you still need All those VM’s for testing the final packaging so install away – you’ll need them. Do the minimal install possible – just what a newbie would install. I installed a Minimal Debian 7.2 (32 bit), SuSu (64) and Fedora 20 (64) in 6 hours. that’s just to get a login prompt and get the guest box additions installed. It took about that time to get Win 7 installed.

Here’s another fact you can’t avoid. Ruby is a pain in the ass to cross compile unless you compile on a system with a running Ruby. Don’t believe me? Try it or skip way ahead when to where I describe the MinGW setup.

I happen to be using Ubuntu 13.10 (64 bit). If you are not running a Debian based distribution then you’ll have extra learning. As you probably know, you can run a 32 bit linux app on a 64 bit system. You can also install 32 bit compilers and 32 bit -dev packages. Or the arm compilers and libs. Don’t do that yet. It’s very confusing and and is just buggy enough to make you think you did something wrong. What does work much better (for debian) is chroot and even better for numb nuts like me is schroot. I’m not going to tell you how to apt-get it or how to set up the .conf files. You are only a few clicks with Google from finding decent blow by blow description of how to set it up. Pay attention to the debootstrap instructions.

I created three [s]chroots at /srv/chroot/ debx86, deb386, debrpi. I’ll talk about debrpi later so ignore that one for now. The first is just enough of a Debian 7.2 file system that when I schroot into it, I can run apt-get and install what ever packages I need like the build-essential (for gcc and binutils), gkt3-dev, and so on. It’s all installed in the chroot and not your running system. Also install ruby (any version whatever the debian default is). No we aren’t going to copy that ruby to build Shoes. We just use it to compile the Ruby that we do want to copy. The [s]chroot environment is sufficiently sandboxed to be really annoying sometimes. Learn to accept it. Do the same schroot dance for the 32 bit version of Debian. Because of the magic of muli-arch being able to run 32 bit apps on a 64 bit host, you’ve installed 32 bit compilers, headers & libraries in the schroot and you’ve haven’t cluttered up your booting Linux. Yes, if you’re paying attention it’s the almost the same amount of work if you just installed the dev tools in each of your VM’s except you don’t waste 5 or 10 minutes logging in to a VM with a deep desire to strangle the pointy haired person who decided what menu’s and command prompt are used for SuSe and Fedora and curse the odd way VirtualBox’s shared folders is damn near useless unless you sudo (assuming sudo is installed). There is no need to deal with those annoyances in a chroot.

I like schroot over chroot because it mounts /home and stuff in the chroot for me so I can just cd into my Shoes3.2 dir and run my rake tasks to build, package and edit the files. You do have to set your shell PATH up each time to remove the /home/yourname/rvm because that ruby won’t run in a chroot. Confused? Of course you are. I suggest you try setting up a chroot and doing the debootstrap and apt-get dancing – eventually the light bulb will go off and you’ll understand.

Before I discuss the raspberry pi schroot or the MinGW setups this would be a good time to back out of the weeds and remember why we went in there.

Dependencies.

Shoes has dependencies. You can’t run Shoes with a ruby lib and a jpeg lib and a [un]gif.lib. You might need a curl lib and of course you need the graphics libs. But there’s more – Ruby has needs to. Perhaps openssl and yaml would be useful And ffi and dbm. It would be nice if the end user of your binary shoes has installed exactly the same libraries that you have. As we used to say, when I was growing up, “People in hell want ice water, too”. There is not a chance in hell that the end user will have what you want and certainly not the versions you want. On the bright side, sometimes it’s not that big a deal because you can bundle the needed libs (dependencies) You absolutely have to do this for Windows. You have to include some of them for Linux. Which ones? That is the $64,000 question.

We know we have to include a properly built libruby and whatever libs it wants so we have to have them to copy. Ruby dependencies tougher. You’ll have to (cross) build or install them before you compile Ruby otherwise the Ruby build thinks you don’t want the missing libs. You have to watch ruby being built and check the logs to see what it didn’t find. It’s up to you if they go missing.
I don’t most Shoes folks would miss dbm (and cousins ndbm, sdbm) but then I don’t know much, so I built (apt-get install in schroot) all the Ruby dependent libs except curses and Tcl/Tk – Those would be counter to Shoes. Then I created a little shell script that configures the Ruby build. You need a script of your own because you’ll run it several times and then clone/modify it for the next thing to build. Of course you need to download the ruby source code. I put it and my script in /opt/src {chroot}/opt/src

I build the Ruby binaries to live in /usr/local (–prefix) because (1) you need the chroot installed ruby if yours fails to run. It could happen. Inside my chroot I have /usr/local/ first in my $PATH. It’ll make sense once you’ve played with chroot. Note: you have to install your ruby to /usr/local as root so su, or login to the schroot as root to run ‘make install’. You also have to root in the chroot to run apt-get. Just like a real system.

Here’s the script I use to configure Ruby (i386, x86_64 & raspberry)

#! /bin/bash
# copy this to the ruby dir or soft link it.
./configure --enable-shared --enable-load-relative
--disable-install-doc
--prefix="/usr/local"

After that, you apt-get install all the other dependencies (gtk2-dev, etc) into the schroot. Then setup the Shoes Rakefiles for the architecture to want to build. Modify the appropriate make/{arch}/env.rb with the values you need. rake, edit, rake clean, repeat until it it compiles and links everything. Each architecture choice in the Shoes rake builds into a different subdirectory. Look at the crosscompile file to see which one is setup at the moment. You can tar that up and move to the VM or wherever you what for a test. Hint: if you ‘export DISPLAY=:0.0’ when inside the chroot you may be able to test your 64 and 32 bit linux builds.

PLEASE NOTE: THERE IS ALTERNATIVE CODE in the env.rb files because I didn’t always build things
the way I’ve described. Some of the code is useless and some of it it is left as hints of what you can do.

Raspberry Pi (Raspbian)
You need to do some Google searching to find the instructions for setting up a schroot with raspian and the Qemu emulator. You’ll have to build or apt-get Qemu first and install it properly before you do the debootstrap and apt-get install ruby and all the dependencies. It’s surprisingly easy. Then build your ruby properly. Be aware: running gcc and ruby under the emulator is slow – it takes about the same amount of wall clock time as doing it on the raspberry itself.

Also note that for the raspberry, I don’t login into the chroot and build shoes inside the chroot. Instead, I installed the Ubuntu arm compiler after verifying that it will use the hardfp capabilities of the raspberry. I used that to compile and build shoes. So I only used the chroot to build ruby and install all the dependencies of ruby and shoes. In the env.rb I rewrite the paths returned by pkg-config to point to the longer paths into the chroot. For the Raspberry, it’s a mix of chroot work then using the ‘big’ system. For bonus points, once you get to build and link, you can chroot in and then run Shoes under the emulator (set DISPLAY) and it runs! That’s far more magic than we should expect.

MinGW
MinGW cross compiles is sufficiently complicated that I’ll start a new blog post.

Leave a Reply

Your email address will not be published. Required fields are marked *