Monthly Archives: March 2014

Shoes Command Line

Did you know that Shoes has a command line? It’s very useful. While this posting is about Shoes 3.2 (Federales) on Linux, It’s there for Windows and OSX users of any version of Shoes if you know the path of the installed Shoes app. There are several ways to get Shoes for your Linux system. The simple way is to
download a binary distribution, a .run from here.

I’m going to use the Raspberry Pi version of Shoes 3.2b4 on a newly created Raspbian sdhc card. The only changes I’ve made to the basic installed Raspbian is to replace the ‘pi’ user with ‘ccoupe’ and get ssh working. Shoes is a GUI application so you should startx and open a LxTerminal. First, download the run file and make it executable.
ccoupe@pi ~ $ wget http://shoes.mvmanila.com/public/shoes/shoes-3.2b4-gtk2-armhf.run
ccoupe@pi ~ $ chmod +x shoes-3.2b4-gtk2-armhf.run

Raspbian has a cute little feature-bug with ‘su’ which is used by the installion script because not all systems have sudo installed. There are two ways to handle the bug. One way is to set a password for ‘root’ like this:

ccoupe@pi ~ $ sudo passwd root
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully

Or you can just hand edit the and copy the file that needed su used for. I also show that method below because it also shows how to create menus for Linux desktop menus.

ccoupe@pi ~ $ ./shoes-3.2b4-gtk2-armhf.run
Verifying archive integrity... All good.
Uncompressing Shoes....................................................................................................................................................................................................................................................
Shoes has been copied to /home/ccoupe/.shoes/federales. Need root password
to copy Shoes.desktop to /usr/share/applications
Password:
su: Authentication failure

Oops. We’ll work around that failure. It’s obvious what it wanted to do. Shoes was installed in ~./shoes/federales. Lets go there and poke around.
ccoupe@pi ~/.shoes/federales $ ls
CHANGELOG.txt libcurl.so.4 libshoes.so shoes
COPYING.txt libgif.so.4 libssl.so.1.0.0 shoes-bin
debug libjpeg.so.8 libungif.so.4 Shoes.desktop.tmpl
fonts libruby.so libyaml.so shoes-install.sh
lib libruby.so.2.0 README.txt static
libcrypto.so.1.0.0 libruby.so.2.0.0 samples
ccoupe@pi ~/.shoes/federales $ more shoes-install.sh
#!/bin/bash
#pwd
ddir=$HOME/.shoes/federales
#echo $ddir
mkdir -p $ddir
cp -r * $ddir/
sed -e "s@{hdir}@$HOME@" Shoes.desktop
echo "Shoes has been copied to $ddir. Need root password"
echo 'to copy Shoes.desktop to /usr/share/applications'
su root -c 'cp Shoes.desktop /usr/share/applications'

That’s the script the install process ran at the end. It justs rewrites a template with sed and copies that. I’ll do that by hand.
ccoupe@pi ~/.shoes/federales $ sed -e "s@{hdir}@$HOME@" Shoes.desktop
ccoupe@pi ~/.shoes/federales $ sudo cp Shoes.desktop /usr/share/applications/

Verify there is a Shoes entry in the Programming menu. Since Shoes is not installed in the system directories (except for the .desktop file), you can’t hurt the system by redoing the install. So Lets get to the commandline.

./shoes -h
Usage: shoes [options] (app.rb or app.shy)
-m, --manual Open the built-in manual.
-p, --package Package a Shoes app for Windows, OS X and Linux.
-g, --gem Passes commands to RubyGems.
--manual-html DIRECTORY Saves the manual to a directory as HTML.
--install MODE SRC DEST Installs a file.
--nolayered No WS_EX_LAYERED style option.
-v, --version Display the version info.
-h, --help Show this message

-p and -m are the same as the Shoes splash screen links. You could Make a ‘Shoes Manual’ menu entry with what I’ve shown you. Just copy and modify that .desktop file. That’s how Windows gets Three menu entries for Shoes.

Shoes has it’s own private Ruby interpreter and gem handling. -g or --gem is what I really want to explore but there is a an undocumented --ruby option you should know about first. Everything after –ruby is passed on just like a normal Ruby command line.
ccoupe@pi ~/.shoes/federales $ ./shoes --ruby -e "puts RUBY_PLATFORM"
armv7l-linux-eabihf

You won’t see it on a fast system but calling ruby that way can be slow. Useful but limited. Your chances of processing a rake script that way is very low. I only bring that up because Shoes will start another copy of itself just that way in order to compile some gems. That gets us back to gems, -g or –gems

Gem are Ruby Libraries or Extensions that are downloaded from the net and installed into your Ruby and when your script needs that, you can require 'gem-name'. Python has something similar as does Perl. There are lots of gems that Shoes users might want to use. Older versions of Shoes used to include a couple of gems, Hpricot for parsing xml and Sqlite. Shoes 3.2 does not include them currently. It’s OK. I’ll explain a better way in a minute.

Gems are pure Ruby code or a mixure of Ruby & C source code. If you get one with C source code, it needs to compiled before in can be installed. Note: newer gems may have precompiled C but you can’t depend on that being true for the gem you want. To compile ‘C’ you need the compiler and build tools. On the Raspberry, these are already installed.

Lets see what the Gem environment looks like with -g env.
ccoupe@pi ~ $ ./.shoes/federales/shoes -g env
RubyGems Environment:
- RUBYGEMS VERSION: 2.2.2
- RUBY VERSION: 2.0.0 (2014-02-24 patchlevel 451) [armv7l-linux-eabihf]
- INSTALLATION DIRECTORY: /home/ccoupe/.shoes/+gem
- RUBY EXECUTABLE: /home/ccoupe/.shoes/federales/shoes --ruby
- EXECUTABLE DIRECTORY: /home/ccoupe/.shoes/+gem/bin
- SPEC CACHE DIRECTORY: /home/ccoupe/.gem/specs
- RUBYGEMS PLATFORMS:
- ruby
- armv7l-linux
- GEM PATHS:
- /home/ccoupe/.shoes/+gem
- /home/ccoupe/.gem/ruby/2.0.0
- /home/ccoupe/.shoes/federales/lib/ruby/gems/2.0.0
- GEM CONFIGURATION:
- :update_sources => true
- :verbose => true
- :backtrace => false
- :bulk_threshold => 1000
- REMOTE SOURCES:
- https://rubygems.org/
- SHELL PATH:
- /usr/local/sbin
- /usr/local/bin
- /usr/sbin
- /usr/bin
- /sbin
- /bin
- /usr/local/games
- /usr/games

Right off, we see some interesting things. The version of RUBYGEMS is pretty current as is the Ruby Version. Don’t worry that armv7l-linux-eabihf doesn’t match anything you think is raspberry like. It’s just Ruby being ‘different’. Gems will be installed in ~/.shoes/+gem .

Let’s see if we have any gems:
ccoupe@pi ~/.shoes/federales $ ./shoes -g list --local

*** LOCAL GEMS ***

bigdecimal (1.2.0)
io-console (0.4.2)
json (1.7.7)
minitest (4.3.2)
psych (2.0.0)
rake (0.9.6)
rdoc (4.0.0)
test-unit (2.0.0.0)

Those gems are included in Ruby 2.0. They aren’t in ~/.shoes/+gems. Notice that there a multiple GEM_PATHS.
Ruby is flexibile that way — there always two or more correct ways to get things done. Let’s install a pure ruby gem ‘metaid’. I don’t remember what the gem does, just that it’s small and pure ruby -its only a example. Be patient, Shoes on Pi + downloading can take a while.
ccoupe@pi ~/.shoes/federales $ ./shoes -g install metaid
Fetching: metaid-1.0.gem (100%)
Successfully installed metaid-1.0
Parsing documentation for metaid-1.0
Installing ri documentation for metaid-1.0
1 gem installed
Exiting RubyGems with exit_code 0

Try the -g list –local again. There it is:
...
json (1.7.7)
metaid (1.0)
minitest (4.3.2)
...

How about a gem that has ‘C’ code to be compiled. Serialport is something Raspberry pi folk might be interesting in using. It allows to you read/write from a /dev/ttyxxx. Again this only an example for this post. I’m not claiming it’s what you want.

ccoupe@pi ~/.shoes/federales $ ./shoes -g install serialport
Fetching: serialport-1.3.0.gem (100%)
Building native extensions. This could take a while...
Successfully installed serialport-1.3.0
Parsing documentation for serialport-1.3.0
Installing ri documentation for serialport-1.3.0
1 gem installed
Exiting RubyGems with exit_code 0

Note that ‘Building native extensions’ line. Depending on the gem, it can take a long time on the Raspberry pi. Gems can also fail to build if you don’t have the dependent libraries and header files. Want to see that? Let’s go, we can learn from out experiments. First we’ll get a list of possible sqlite gems to install:
ccoupe@pi ~/.shoes/federales $ ./shoes -g list --remote sqlite

*** REMOTE GEMS ***
....
sqlite3 (1.3.9 ruby x64-mingw32 x86-mingw32 x86-mswin32-60)
sqlite3-dotnet (3.7.2.1)
sqlite3-ironruby (0.1.1)
sqlite3-ruby (1.3.3, 1.3.2 x86-mingw32 x86-mswin32-60, 1.2.5 x86-mswin32, 1.2.3 mswin32)
....

There is a great example of gems with multiple precompiled binaries inside and more that one choice that Ruby allows. I want the sqlite3. Houston, we have a failure!
ccoupe@pi ~/.shoes/federales $ ./shoes --gem install sqlite3
Fetching: sqlite3-1.3.9.gem (100%)
Building native extensions. This could take a while...
ERROR: Error installing sqlite3:
ERROR: Failed to build gem native extension.

/home/ccoupe/.shoes/federales/shoes --ruby extconf.rb
checking for sqlite3.h... no
sqlite3.h is missing. Try 'port install sqlite3 +universal',
'yum install sqlite-devel' or 'apt-get install libsqlite3-dev'
and check your shared library search path (the
location where your sqlite3 shared library is located).
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
....
extconf failed, exit code 1

Gem files will remain installed in /home/ccoupe/.shoes/+gem/gems/sqlite3-1.3.9 for inspection.
Results logged to /home/ccoupe/.shoes/+gem/extensions/armv7l-linux/2.0.0/sqlite3-1.3.9/gem_make.out
Exiting RubyGems with exit_code 1

That’s almost too easy. It even tells us to we don’t have sqlite3.h and to ‘apt-get install libsqlite3-dev’ on the pi. More difficult errors might require looking the /home/ccoupe/.shoes/+gem/extensions/armv7l-linux/2.0.0/sqlite3-1.3.9/gem_make.out

Let install sqlite3-dev
ccoupe@pi ~/.shoes/federales $ sudo apt-get install libsqlite3-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
sqlite3-doc
The following NEW packages will be installed:
libsqlite3-dev
0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded.
Need to get 537 kB of archives.
After this operation, 1,175 kB of additional disk space will be used.
Get:1 http://mirrordirector.raspbian.org/raspbian/ wheezy/main libsqlite3-dev armhf 3.7.13-1+deb7u1 [537 kB]
Get:2 http://mirrordirector.raspbian.org/raspbian/ wheezy/main libsqlite3-dev armhf 3.7.13-1+deb7u1 [537 kB]
Fetched 83.0 kB in 2min 2s (680 B/s)
Selecting previously unselected package libsqlite3-dev.
(Reading database ... 62359 files and directories currently installed.)
Unpacking libsqlite3-dev (from .../libsqlite3-dev_3.7.13-1+deb7u1_armhf.deb) ...
Setting up libsqlite3-dev (3.7.13-1+deb7u1) ...

and try to install that gem again.
ccoupe@pi ~/.shoes/federales $ ./shoes --gem install sqlite3
Building native extensions. This could take a while...
Successfully installed sqlite3-1.3.9
Parsing documentation for sqlite3-1.3.9
Installing ri documentation for sqlite3-1.3.9
1 gem installed
Exiting RubyGems with exit_code 0

There you go! No magic required. In true Ruby fashion there is another way to install gems from inside a Shoes script (see sample/simple-rubygems.rb) but it won’t install or compile what you don’t have so you’d end up confused about the errors and end up doing what I just demonstrated.

Perhaps you’re not using a Pi. It’s the same for for 32bit or 64 bit Linux (Suse, Fedora, Debian, Ubuntu…) The only difference is which Shoes you download and what commands you need to install the compiler and build tools – port,yum,apt-get and what is the name of the package that has what you want. Sadly. That’s more difficult than I can write up on a Shoes post. I will mention in passing that a bare Ubuntu needs ‘build-essential’ – search for it – it might be build-essentials.

Good Luck and enjoy your Shoes!

Thinking more about Packaging

Now that I’ve refreshed my memories of how packaging works and some exploratory hacking with cgi.rb at this website, I can think more about how it should work. It’s very tempting to just add some additional items to the Pack GUI screen for armhf/i686/x86_64 and add a [arch] parameter and the only thing that changes is using this website for downloads instead of the dead shoes.heroku.com. That isn’t as easy I make it sound. It’s also not very elegant.

Ideally, I’d want the Pack Screen of Shoes to download a list of available Shoes distributions and build the GUI from them and the only option is to package to what’s available. That will take some surgery on pack.rb and I’m going to have a cgi script compute what’s available on the website so I can just upload a new version and not have to remember to find all the static pages and files. Weather Forecast: Cloudy with a chance of fail.
Here is the cgi that Pack.rb will ask for, equivalent to a
wget http://shoes.mvmanila.com/public/select/pkg.rb

#!/usr/bin/ruby
lns = []
Dir.glob('../shoes/*') do |fp|
tm = File.mtime(fp).to_i
sz = File.size(fp).to_i/1048576
fn = File.basename(fp)
lns << "#{sz} #{tm} #{fn}" end require 'cgi' cgi = CGI.new cgi.out{ lns.join("n") }

That produces a (wget) file:
12 1394777130 Shoes-3.2b3-msw.exe
17 1394775751 shoes-3.2b3-gtk2-armhf.run
16 1394776193 shoes-3.2b3-gtk2-i686.run
17 1394776919 shoes-3.2b3-gtk2-x86_64.run
17 1395463602 shoes-3.2b4-gtk2-armhf.run
17 1395463956 shoes-3.2b4-gtk2-i686.run
17 1395464575 shoes-3.2b4-gtk2-x86_64.run
12 1395463307 shoes-3.2b4-msw.exe
10 1394007512 shoes3.2b3.tar.gz

Yeah, I could have computed file size in MB at the Shoes Client. I could also write the cgi in python, or TCL or bash. I include the mtime so the client (Shoes/pack.rb) can figure out which version of Shoes3.2 is newer because you can't depend on The Cecil's naming scheme. I can't sort by name. Is Shoes-3.2b4 newer than Shoes-3.2rc1? or just 3.2?. This one of those choices that you know will bite you in the butt, either way. I leave it to pack.rb to parse and display the servers list. Yes, parsing the file names will be more arbitrary than a control freak would like.

Shoes 3.2b4

I fixed a couple of bugs. Whee! For the binject extension. Whoa! That’s dangerous territory to be exploring, dude! Indeed it is, but with binject included then Shoes doesn’t cough up a hair ball when selecting packaging from the main screen. Granted the splash screen says “obsolete” but that could mean ‘not working yet’. It could work from and for Linux. I think it should work for Raspberry pi users and if I can do that, I can do it for the other Linux – X86_64 and i686.

Shoe 3.3b4 is here I doubt it fixes anything most people care about.

If I wanted to fix (Red) Shoes packaging I should enumerate the current fails and how I would fix them. Packaging has always been on my mind. Notice how the website has longer urls and the files names have more info embedded than raisans.exe or policeman-novideo.run? Of course there is a reason for the extra info. I recognized the problem back when Shoes/OSX had ppc and intel builds. Same problem for arm/intel builds. You can’t have one .run/.exe/.dmg for download that can deal with all OS/arch variations.

I do have Linux binary downloads that work. The packager is just too crusted to deal with variations of Linux (or OSX or Windows). When shoes.heroku.com went into permament maintence mode (dead) it also broke any chance of 3.1 packaging working. We knew that then. Yes, there are 3.1 downloads at github if you can find them but the Shoes code (native/stubs) was never updated to know that. Nor does Shoes 3.2 know about my 3.2 downloads at this website. Fixable? Yes. Ripe for dream like cloud over-engineering and psuedo abstraction? Yup.

Maybe I’ll include nokogiri and sqlite3 in Shoes 3.2b5 first. That might be a whole lot easier.

Shoes, extconf.rb, mkmf.rb and binject

[UPDATED: March 19, 2014 and March 20]
There’s only a few people in the world that cross compile Ruby as an embedded (dependent lib) for their larger app like Shoes. For a damn good reason – it’s a lot harder than it should be. One reason is extconf.rb tied up with RbConfig::CONFIG in mysterious ways. It requires magic and no one talks about it once they know the secret words. It’s not for malice or laziness – it’s just once you understand it enough to work for you, you move on to fun things.

That’s what I did when I got Shoes (Ruby) extensions ftsearch and chipmunk compiled and cross compiled. I’m tempted to move on myself but there might be another person struggling that needs a hint. extconf.rb and mkmf.rb are dependent on RbConfig::CONFIG vars for building the tests like have_library("z")
and the settings in RbConfig::MAKEFILE_CONFIG are used to create the Makefile that will build/cross compile the extension. Two different things. [Update Mar 18, 2014]. RbConfig::MAKEFILE_CONFIG is mostly in control of what mkmf does. You probably want them to be the same but in a cross compile situation you want them (both) to be something different from the Ruby that is running the extconf.rb. Magic?

For Shoes purposes, binject cross compiles for raspberry pi (arm) x86_64 and i686 builds. Mingw cross compile requires slightly more fun and magic but I’m sure it can be done. [March 20, 2014] Got it. MinGW
version cross compiles binject.

LET ME BE CLEAR: binject and packaging don’t work in 3.2 any better than it does in 3.1 (i.e. not at all) and its useless for most folks until the Windows version of Shoes is working and that may never happen.

Well, this is embarassing. Newer 3.2b3.

After much head scratching and moaning about ftsearchrt, UTF-8 and encroaching socialism in the USA (the usual suspects). It turns out that the manual problem is fixed. It was fixed a month ago but global warming activists deleted that fix. Or I dropped a commit or something. Anyway, I’ve updated the Shoes b3 downloads. It’s not really enough of a fix to be called b4. I did make a minor enhancement to the display of samples to run from the Manual and maybe I fixed some symlinking issues and I should have called it b4. But I didn’t because I want b4 to be the last release of 3.2. I tire of the version numbering. Here is the link to download.

The b4 death march will begin in a few seconds. Most Linux users of Shoes won’t notice the changes and it still doesn’t help Windows and OSX. b4 is mostly about my interests. Sure, I’d like to fix the simple-sphere.rb bug for 64 bit Rubies. Without feedback though, I’m going to persue my interests and that is to
whip down a gem setup bug when you do have the tools to build but only when run from the sample page of the manual. I can count the number of people that will find that bug on half a finger. I’m more interested in getting gems and exts better support for cross compiling to raspberry (and binject for raspberry and packaging scripts for raspberry/shoes).

About that beta 4 of 3.2

You know what happens when good intentions meet reality? I do. I wanted to fix the bug that simple-sphere has on 64 bit Linux. It works fine on 32 bit Linux. I can’t find it. In gdb comparisons, everything looks ok and I’ve done a lot of looking at the C code. That first blur 30 trips the bug. Setting it to 26 works. That is not good news to me. The code is kind of strange because I don’t know what a gaussion blur is or what 26 or 30 means. I guess it’s the center point of a rgb schmering (to make up a word). Why not check the Shoes manual?

That’s when reality stepped forward, kicked me to the ground, kicked me once more and is patiently waiting for me to get up before continuing the beating. Blur isn’t in the Shoes manual. There are a number of things not in the manual. I also noticed the manual was behaving strangely. Open the console window and it’s an ASCII-8BIT vs UTF-8 complaint. So my hack with 3.2b2 and ruby 1.9.3 didn’t work for long. I’m disappointed but not that surprized. Hacks are hacks. Naturally, I went poking around in ftsearchrt.c. There are many things in there that just don’t work it a unicode world. The sort code is just the beginning (and it can sort oddly – I’ve seen that). One solution would be to update all the C code to use wide characters or utf variants for string handling. Ick. Even worse, there are FIXME and TODO comments about changes to 64 bit that need to be done. I really don’t like that path.

Most of it is just passing back and forth with RubyVM API. Perhaps pure ruby would work? So I looked at what the Shoes4 folks have done with the manual. Indeed, it’s pretty much along the lines of what I was thinking I would have to do. Once you discover that the manual is samples/simple-manual.rb it displays OK (garish colors, but it lays out OK and the colors should be tuneable). Sadly, the search function runs out of memory (default jruby alloc) or coughs up an exeception. I filed bug #604 because I’m a team player. Right? Should be fixable. It’s a good place for me to start with.

From a Red Shoes 3.2 perspective it would require distributing Nokogiri as a Shoes gem since it has a native compile component. It’s like the hpricot gem that Shoes used to distribute but I don’t in 3.2. Welcome back. That gem dependency is no big deal for building Shoes from source on Linux so I can test the newer manual code my default dist build without have to cross compile an extension/gem. I might need to change the names of files and maybe some modules but hopefully anything I learn/fix will be transferrable to Shoes4. And vice versa. I like that their manual is (mostly) just a Shoes app. I really like the idea of ditching ftsearchrt. Does it sound like a lot of work? Yes, it is a lot of work and could conflict with Shoes4. Is it the right path? I think it might be.

Shoes 3.2b3 is available

I don’t anticipate another beta for Shoes 3.2 Federales. Yes, I could do more but I think I should pause. Unless I find a show stopping bug, Oh Hell I did find one. Never mind. check back later

[update]
After some digging in the bug, I’m going to release the b3. The files can be download from
here

The bug does need to be fixed but it’s not a show stopper. I’ll describe the conditions where it appears. You have to download and a Shoes 3.2 .run file for your architecture – not builing from source, and from the Shoes manual, you start the rubygems example it appears to fail after building native extensions (it will always fail if you don’t have gcc, make and friends intalled). Appears to fail. Actually it built the gem just fine. Re-run the the example app – no error because it’s installed.

If you tell Shoes to run the samples/simple-rubygems.rb script from the command line or the splash screen “Open an App” it also builds and installs the gem without drama although it’s gawd awful slow on a raspberry pi. It’s not a bug that prevents Shoes.setup from working.

There’s another long standing bug I should fix for b4. samples/simple-sphere.rb fails on 64 bit rubies. That bug is in the C code for blur (I think).

Slightly new website

Welcome to yet another blog about Shoes,  the cross platform GUI programming system based on Ruby.  I’ll also talk about the Raspberry pi because it’s very much in the Shoes spirit.

Some older posts may have links to my  website that used to host them.  I might fix them but if I were you,  I wouldn’t expect that.  Another aside: I’m not a website designer and I don’t care to be.  Getting my blog postings shuttled around is just something that needs to be done before I declare Shoes 3.2 is done enough.