Monthly Archives: October 2014

What’s new in Shoes 3.2.16

Yes, Shoes 3.2.16 is available for download here. For Linux (x86, i386, armhf), Windows 7+, and OSX (Mavericks and Yosemite).

What’s new in 3.2.16?
1. Fixed a nasty hanging bug on Windows where Shoes would appear to hang, usually in the background. This bug has been known since 3.1. It actually consumed 100% of a cpu core and if the system got busy doing other things it wouldn’t empty the event queue so Windows decided it was hung. It wasn’t. I believe it’s fixed now.

2. Shoes can now package a script (or a directory by making a shy of it) and you can
choose to have it packaged “Download if needed”. This option adds 10 to 60KB to your script instead of 8 to 15MB. When it run on the user’s system it downloads and installs Shoes if it’s not already installed. Any platform can package for any platform. Subsequent running of the package is fast since Shoe is installed. It is the default option.

What’s next?
Both of these features took a lot of work to fix and to implement so minor bug fixing has been on the back burner. There is an OSX bug with radio buttons (easily seen in the packager). There hundreds of compiler warnings that don’t need to be there. There’s a lot of How-To’s that could be written. Shoes Windows Gtk could have a Theme manager so the widgets might look a bit more Window’s like IFF it can be run from Shoes. I’d like to add some additional capabilities to the Packager. No promises, yet.

One OSX step at a time

I cloned and modified stub.m into a creature called osxdnl.m for use when packaging ‘download if needed’ (dnlif). stub.m assumed it was download a .dmg which it mounted and copied to /Applications. Shoes 3.2 (federales) uses a .tgz for the download Shoes.

When you package ‘myapp.rb’ or ‘myapp.shy’ with dnlif option for OSX you’ll create an myapp-osx.tgz. You send that to your friends & co-workers or upload it to your website. It’s a small file. The end user double clicks the myapp-osx.tgz, OSX offers to expand it and you get a Myapp with the shoes icon. Double click that and if Shoes is installed in /Applications it will start running myapp.rb or myapp.shy

If Shoes is not installed, then it downloads and installs Shoes. Thats how it works in Linux and Windows and soon on OSX (I hope). It’s the default way to package in the upcoming Shoes 3.2.16. You can still include Shoes with your package but it’s not the default. dnlif is much simpler for all. A working GUI downloader in Objective-C/cocoa that I can run from a bash script is the first step. It should proceed quickly. Unless it doesn’t.

Doh! Hang be gone

Three of us have reported success with running the 3.2.16 beta version for Windows. You can never say never, particularly when it’s a threading problem. As I noted previously, Gtk File descriptors (fd) are not the same as Ruby fd FOR WINDOWS. After chasing down the rabbit holes and inventing failing methods for moving from one rabbit warren to another I had one of those blindingly obvious in retrospect insights.

I don’t have to run my own poll loop on Windows. In fact, the pool loop on Linux is probably there for the days gone by of Green Threads. Those days won’t be coming back. Every one has native threads and multiple cores (except my Raspberry pi) Let the default GTK thread do it’s thing which synchronously calls most of the Shoes code. It’s only additional Ruby threads initiated by Shoes (rarely) or the users script that makes threading calls that needs a wake call to run and I can trigger that with a gtk_timeout function that calls rb_thread_schedule().

CPU dropped from 100% to negligible. The hang disappeared and we all live happy in the land of milk and honey dancing polka’s with the unicorns and leprechauns.

Obviously this will be the major bug fix for the upcoming Shoes 3.2.16 but I want to finish restoring the packager before a general release of 3.2.16 – just some osx work left – how many days could that take?

If you can’t wait for the Windows hang fix, you can find the beta directory from the downloads tab at the top of the screen.

What the hell does select() do?

Warning – This could be wrong

And no guarantees, either. I’m thinking out loud, exposing my ignorance.

Perhaps I should read up on what select() is intended to do: If none of the fd’s File Descriptors (which may be threads) has anything to do, then the process is blocked until one of them (which may be a thread) has data available to read/write/exception (aka error or timeout) and then number of fd’s passed in to select that wouldn’t block on a read/write op is returned. Think read for illustration purposes.

I prefer not to know the gritty details of how that happens when my mouse moves, but that fd would have data that is ready to read so if select was blocked waiting for mouse movements or keyboard clicks or TCP-IP packet arrival it would unblock the process and report to the select() caller that one of the watched fd’s is ready to be read (for illustration).

Thats what the shoes_app_g_poll does. It takes a set of Gtk fd’s (read set, write set, exception set) and if none of them has anything to do it tells the OS to not run me until something happens on those fd’s. So it waits for a mouse click or a key-press. Then the caller has to sort out which event happened and call the Shoes code needed to respond to it. There’s a lot code to make that translation from mouse click to executing the { block} attached to a button in your script. That works. It has worked for years and years in Linux. Native Windows and OSX are slightly different but they do the same thing: call some Shoes code. Thats the 10,000ft view.

Now we need to look at what Shoes/GTK/Windows does instead of the Shoes/GTK/Linux that just works. For Windows, Shoes/GTK only works for a few minutes until Windows in it’s mommy loving wisdom decides the application is hung and “not responding”. It also use 100% of 1 cpu core doing nothing which is anti-social. Even though you might thing its working just fine for a while, Mommy thinks its hung.

If you had read all the documentation and source code then you’d know that GTK/Windows file descriptors aren’t really like Linux fd’s. And you know the Ruby Windows has its own stdio and threading implementations. I didn’t understand that either. As it turns out Ruby/Windows file descriptors (fd) are not really like Linux fd’s either, And they aren’t like GTK/Windows fd’s because that would be too simple. For extra fun they both want to be opaque so programmers don’t have to know the details. Gtk does document it. Ruby? not so much (aka nada)

We’re flying just above ground now, heading for the marsh at the end of the landing strip unless we can get lift. Perhaps you’re thinking that I can’t match Windows/Gtk fd’s to Ruby’s fd (neither of which is like Linux fd’s except in opaqueness). I’m not sure I can move between them either. I’m not sure I want to fix anything for Windows.

Deep in the weeds

I’ve been working on the Shoes hanging bug on Windows. It hasn’t been easy and I’m way above my pay grade on Windows internals. So many rabbit holes to explore. Every time I think I know what’s happening, I’m proven wrong.

Today, once again. I think I know what’s going wrong. I’ve got clues and hypotheses. I’ve confirmed some “that’s not right” behavior deep down in Shoes and Gtk/win32. I’m also smart enough to know that I don’t know my ass from a hole in the ground. I’m getting a lot more Windows education than I ever wanted.

At the moment, I think gtk/win32 and Ruby have different ideas about what an ‘fd’ is and that I should ‘unify‘ them. [snort]. One fd table to rule them all. Or a hack so each (Gtk, Ruby) believes their is one unified fd table. I’m leaning towards the hack. And then I’ll fix the real problem.

Or Shoes 4 will be released.

Things that go ‘bump’ in the night – Shoes 3.2.16

I really wanted to produce Shoes 3.2.16 with the download if needed packaging. I did the hard part with Windows, right? Not according to bug reports. They want the Windows Hang problem fixed and one of them offered to help and he knows a boatload more about Windows than I do. He even went so far as to build Shoes (from source) on his Windows box. That’s a serious offer to help! Do not turn down help with Windows. Ever!

So we’ve been studying the hang problem on Windows. It’s not solved and it might be above my pay grade to solve it (I’m mystified)

I need a break from thinking about Threads and C and Windows and Ruby internals. So, I started working on the package Linux ‘download if needed’ code. Some of you might think that would be easy because that always worked. Not according to me. Remember the old days – a downloaded Shoes/Linux might work but it probably wouldn’t. I fixed that in Shoes 3.2 (I think). Installing Shoes if needed and then running the script or shy became a bit more tricky when I added three Linux architectures.

It’s almost working. Close enough to know that will work. The struggle continues.