(when (launch-authorised? president)
  (fuel-missile icbm)
  (open-hatch silo)
  (launch-missile icbm))

Another example of control operator is the very useful and-let* macro, semi-standardised in SRFI-2, tests several expressions in sequence, aborting the computation if any of them evaluates to false, optionally binding variables to the expressions’s resulting values in nested scopes otherwise. If none of the expressions evaluate to false, the body is executed in the resulting lexical environment. Those familiar with Haskell may find it somewhat similar to the Maybe monad.

Actually several of Scheme’s own standard control operators are macros, implemented almost exactly as user-defined macros. A named let can be easily rewritten as a letrec operator:

;; a named let expression
(let collect ((seq (produce-list-of-numbers))
              (even '())
              (odd '()))
  (if (null? seq)
      (cons even odd)
      (let ((num (car seq)))
        (if (even? num)
            (collect (cdr seq) (cons num even) odd)
            (collect (cdr seq) even (cons num odd))))))
 
;; becomes a letrec expression
(letrec ((collect (lambda (seq even odd)
                    (if (null? seq)
                        (cons even odd)
                        (let ((num (car seq)))
                         (if (even? num)
                             (collect (cdr seq) (cons num even) odd)
                             (collect (cdr seq) even (cons num odd))))))))
  (collect (produce-list-of-numbers) '() '()))

Macros are of course not limited to adding new control operators to a language. One can design his own little language for solving specific problems, like stocks trading. Or a data modelling language, a graphics drawing language etc. A language for object-oriented programming such as Meroon can be seamlessly embedded within Scheme. Paul Graham’s On Lisp book on advanced Common Lisp programming has several chapters devoted to macros. In short, the developer becomes almost as powerful as the language designer, and such distinction is blurred. Armed with such power, the developer becomes much more productive and, therefore, happy.

Unfortunately not many programming languages provide good macro facilities. The C preprocessor, for instance, only does textual replacement and is no aware of expressions. Granted, the syntax of the programming languages of the Lisp family is very simple and uniform, making better macros possible. Nevertheless other languages like Haskell and OCaml with more complex syntax also provide such facilities.

3.3.1 — Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs.

to

3.3.1 — Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs. Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited).

The intended effect is obviously to ban third-party abstraction layers, like then soon-to-be-released Adobe’s Flash for iPhone. But the wording is very broad and cover too many uses of other programming languages and technologies. Steve Jobs demoed himself the Tap Tap Revenge game for iPhone, which reportedly uses Lua. The backslash was too big to ignore, making Apple revise the agreement again. This time section 3.3.2 was changed from

3.3.2 – No interpreted code may be downloaded or used in an Application except for code that is interpreted and run by Apple’s Documented APIs and built-in interpreter(s).

to

3.3.2 – Unless otherwise approved by Apple in writing, no interpreted code may be downloaded or used in an Application except for code that is interpreted and run by Apple’s Documented APIs and built-in interpreter(s). Notwithstanding the foregoing, with Apple’s prior written consent, an Application may use embedded interpreted code in a limited way if such use is solely for providing minor features or functionality that are consistent with the intended and advertised purpose of the Application.

Although still requiring Apple’s prior written approval, it leaves the door open for the use of other programming languages on iOS devices. Apple Outsider believes this is all about Lua, and he may be right. But of course Lua is not the only game in town.

I always thought my use of Scheme in Reverso was ok with the previous version of the license agreement. No code is ever downloaded, and not interpreted either; Gambit-C compiles Scheme code to C which is compiled with Apple’s official tools to create a native library. But this changed with the new 3.3.1 section because my code was not originally written in one of the approved languages. James Long, who first compiled Gambit-C for the iPhone, claimed that Scheme was dead on the iPhone. But then Apple relented and changed section 3.3.2. What about now?

I believe that makes Scheme usable again, if used just like in Reverso: as a library. The application is written in Objective-C, but uses Scheme code (compiled to C) to only add features “that are consistent with the intended and advertised purpose of the Application”. Albeit it can be claimed that the code is not interpreted, this use follows the spirit of the law, even if it does not follow its letter.

“If now, while they are one people, all speaking the same language, they have started to do this, nothing will later stop them from doing whatever they propose to do.” – Genesis XI, v.6

I believe I have got the first Scheme application past Apple review into the iTunes App Store. It is yet another Reversi clone, called Reverso. It is a combination of 90% Scheme and 10% Objective-C, written with Gambit-C Scheme. James Long has already shown how to compile Gambit-C for the iPhone, and I started from there. My Scheme code is compiled to C by Gambit and later by GCC to produce native ARM code, bundled in a static library, which is ok with the iPhone SDK license agreement. The Objective-C then calls the library as a pure C library. The Scheme code deals with position evaluation, alpha-beta pruning, transposition tables, move legality, different strategies and so on. The Objective-C code deals with sound, animations, GUI, user preferences, basically everything that calls the iPhone OS API. Reversi was chosen because I like strategy games and it is much more algorithmic than artistic, and I am no artist.

The performance of the code is excellent. I used some Gambit-specific declarations, only fixnum arithmetic, pre-allocated a large heap, and called the garbage-collector every time the user needed to think. The search is not memory intensive, but the transposition tables are. I did not write a specific hash function but relied on Gambit-C’s table type. The boards used during search were retrieved from a pool (the newest Gambit-C has made subu8vector-move! a public API), so they did not put pressure on the garbage-collector. In the end it was a very successful experiment. Developing with Scheme is orders of magnitude more productive than with most other languages. Gambit-C is also one of the best Scheme compilers out there, and made my job a lot easier.

Update: There was a bug in the application that caused it to play poorly. The weak AI was not Gambit’s or Scheme’s fault, but mine. I already sent an updated version to Apple that will play much better. Besides fixing the bug, I am now using Zobrist’s hashing instead of Gambit’s own hash function, that performs poorly for game positions. It is still written in Scheme, though. Now I am waiting for the approval of the update, and to hear further feedback from my users.

“You think you know when you can learn, are more sure when you can write, even more when you can teach, but certain when you can program.” – Alan Perlis

Just after have read Lisp in Small Pieces I felt the urge to write a Scheme compiler. This sentiment is common in the Lisp community, I guess. There are dozens of Scheme systems, some good, some bad, some maintained, some abandoned. There are large optimising compilers and small interpreters for embedded systems. But I, well, needed to know. I needed to know how to write a program that takes another as input and generates simple instructions that do the same computation. I needed to know how to a write a program that interprets those instructions, a runtime to provide primitives, a garbage collector to manage memory etc. So I started doing some coding in my spare time, and called the monster Sly Scheme.

I failed miserably. I began with the reader, believing that basic I/O was fundamental for a REPL (the REPL was the goal I set myself to achieve). The reader was growing too complex and my interest simply disappeared. Then I focused on the types, creating a large hierarchy of them, only to lose interest again. After some time I came across Abdulaziz Ghuloum’s An Incremental Approach to Compiler Construction, which starts with the compiler itself. Let another Scheme do the I/O! I then took this path, running my compiler with Gambit-C. But instead of generating machine code, it generated instructions for a virtual machine. I then proceeded in lockstep: Wrote in Scheme a new compiler feature, and then in C just enough to interpret the new instructions. Finally most of R4RS Scheme is compilable. Then I wrote a simple stop-and-copy garbage collector with two semi spaces. After that, I needed a runtime with enough procedures to run the compiler without Gambit-C. Almost an year later, I have my REPL (I am not that bad as a programmer, but this is just a hobby that I do in my spare time).

The code now sits in a Github repository. For those who cares to check it out, I must warn that everything was done as simplistic as possible, so I could have a working interpreter ASAP. Some things were even done outright stupidly. The compiler generates too many closures, it fails with simple cases, the VM uses a giant switch, it is slow, the garbage collector too etc. I probably will hack this code for the years to come, even if only as a hobby of mine. But the most important contribution of this project is how much I have been learning from it.

In a large class of problems one is given a start state and some predicate for a desired final state. Moreover, the rules that dictate how a successor state is generated from a previous one are also given. Take a board game for instance, like tic-tac-toe. There is the initial state (which is the empty board), the predicate for a desired state (a state in which tree pieces of mine are aligned), and rules to create one state from another (the rules of the game). Another example of such problems is the “Escape from Zurg” one, that is stated as follows:

Buzz, Woody, Rex, and Hamm have to escape from Zurg. They merely have to cross one last bridge before they are free. However, the bridge is fragile and can hold at most two of them at the same time. Moreover, to cross the bridge a flashlight is needed to avoid traps and broken parts. The problem is that our friends have only one flashlight with one battery that lasts for only 60 minutes (this is not a typo: sixty). The toys need different times to cross the bridge (in either direction):

TOY	TIME
Buzz	5 minutes
Woody	10 minutes
Rex	20 minutes
Hamm	25 minutes

Since there can be only two toys on the bridge at the same time, they cannot cross the bridge all at once. Since they need the flashlight to cross the bridge, whenever two have crossed the bridge, somebody has to go back and bring the flashlight to those toys on the other side that still have to cross the bridge. The problem now is: In which order can the four toys cross the bridge in time (that is, in 60 minutes) to be saved from Zurg?

This class of problems is usually solved by searching. The initial state and all the successors ultimately build a DAG (directed acyclic graph) that is called the search space. Some search spaces are small, like tic-tac-toe playing or even the “Escape from Zurg” puzzle. Others are large, like playing Reversi; even larger, like playing Chess; others are infinite. Needless to say that that are better searching ways than others. This same puzzle, for instance, was solved in Scheme by Ben Simon using the cool amb operator. The amb operator uses first-class continuations to backtrack and take another path in case the first one fails. It was an elegant solution for this puzzle, but it is not adequate for general searching because it backtracks blindly. There are others ways to search blindly, for instance depth-first search and breadth-first search.

In a depth-first search, we analyse the successors of a given state until there are no more successors to analyse, and then backtrack to a previous level of the tree. In a breadth-first search, we analyse the whole fringe of the tree (the horizon nodes) before expanding it one level further using the successor rules. This is good for infinite search spaces, because we will eventually find a solution if there is one. The implementation in Scheme is straightforward:

 
(define (depth-first states successors goal?)
  (if (null? states)
      #f
      (let ((state (car states)))
        (if (goal? state)
            state
            (depth-first (append (successors state)
                                 (cdr states))
                         successors
                         goal?)))))
 
(define (breadth-first states successors goal?)
  (if (null? states)
      #f
      (let ((state (car states)))
        (if (goal? state)
            state            
            (breadth-first (append (cdr states)
                                   (successors state))
                           successors
                           goal?)))))

This is actually all we need for simple searches. Of course we need to supply the initial state, the procedure that generates successors and the goal predicate. These are specific for each kind of problem. Let’s see the Zurg problem procedures:

 
;; the type of the states found in our puzzle
(define-type state
  (previous unprintable:)       ;; previous state
  near                          ;; toys in near shore
  far                           ;; toys in far shore
  flashlight                    ;; location of flashlight
  time)                         ;; time consumed so far
 
(define (start-state)
  (make-state #f
              '(buzz woody rex hamm)
              '()
              'near
              0))
 
(define (final-state? state)
  (and (null? (state-near state))
       (<= (state-time state) 60)))
 
;; at most two toys can travel across the bridge
;; the flashlight must be used in all crossings
(define (successors state)
  (let ((orig (if (eqv? (state-flashlight state)
                        'near)
                  (state-near state)
                  (state-far state)))
        (dest (if (eqv? (state-flashlight state)
                        'near)
                  (state-far state)
                  (state-near state))))
    (map (lambda (toy-pair)
           (let ((toy1 (car toy-pair))
                 (toy2 (cdr toy-pair)))
             (make-next-state state
                              (remove toy2
                                      (remove toy1 orig))
                              (ucons toy1
                                     (cons toy2 dest))
                              (max (time-cost toy1)
                                   (time-cost toy2)))))
         (product orig orig))))

The previous procedures use some utilities:

 
(define (make-next-state state orig dest time)
  (if (eqv? (state-flashlight state) 'near)
      (make-state state orig dest 'far (+ (state-time state) time))
      (make-state state dest orig 'near (+ (state-time state) time))))
 
;; the time each toy takes to cross the bridge
(define time-cost
  (let ((costs '((buzz . 5)
                 (woody . 10)
                 (rex . 20)
                 (hamm . 25))))
    (lambda (toy)
      (let ((pair (assv toy costs)))
        (and pair
             (cdr pair))))))
 
(define (ucons a b)
  (if (memv a b)
      b
      (cons a b)))
 
(define (product lis1 lis2)
  (let lp1 ((lis1 lis1)
            (res '()))
    (if (null? lis1)
        res
        (let ((i1 (car lis1)))
          (let lp2 ((lis2 lis2)
                    (res res))
            (if (null? lis2)
                (lp1 (cdr lis1) res)
                (let ((i2 (car lis2)))
                  (if (member (cons i2 i1) res)
                      (lp2 (cdr lis2) res)
                      (lp2 (cdr lis2) (cons (cons i1 i2) res))))))))))

Our successors procedure does not take time in account. So it is possible for a branch of the tree to go on indefinitely. To guarantee that we will arrive at a solution, we use a breadth-first search:

 
(define (print-path state)
  (let loop ((state state)
             (path '()))
    (if state
        (loop (state-previous state)
              (cons state path))
        (for-each println path))))
 
(print-path (breadth-first (list (start-state)) successors final-state?))

With the result:

#<state #14 near: (buzz woody rex hamm) far: () flashlight: near time: 0>
#<state #15 near: (rex hamm) far: (buzz woody) flashlight: far time: 10>
#<state #16 near: (woody rex hamm) far: (buzz) flashlight: near time: 20>

#<state #17 near: (woody) far: (rex hamm buzz) flashlight: far time: 45>
#<state #18 near: (buzz woody) far: (rex hamm) flashlight: near time: 50>
#<state #19 near: () far: (buzz woody rex hamm) flashlight: far time: 60>

If we change our successors procedure a little, we can use depth-first search:

 
(define (successors2 state)
  (let ((orig (if (eqv? (state-flashlight state)
                        'near)
                  (state-near state)
                  (state-far state)))
        (dest (if (eqv? (state-flashlight state)
                        'near)
                  (state-far state)
                  (state-near state))))
    (filter (lambda (s)
              (<= (state-time s) 60))          ;; filtering out bad states
            (map (lambda (toy-pair)
                   (let ((toy1 (car toy-pair))
                         (toy2 (cdr toy-pair)))
                     (make-next-state state
                                      (remove toy2
                                              (remove toy1 orig))
                                      (ucons toy1
                                             (cons toy2 dest))
                                      (max (time-cost toy1)
                                           (time-cost toy2)))))
                 (product orig orig)))))
 
(print-path (depth-first (list (start-state)) successors2 final-state?))

The advantage of changing the successors procedure is that depth-first searches are usually faster than breadth-first searches, in this case 483ms vs. 24630ms. Also, it uses less memory. So it is always a good idea to try to cut the search space as soon as possible.

• Work with very clever people or alone, from home
• Work with good development tools, i.e., Git, Mercurial, Emacs etc.
• Work with good programming languages, i.e., Scheme (first choice), Haskell, Common Lisp or Lua

These goals are intended to guarantee my long term happiness and stress-free high productivity. I mean a rewarding professional life. So far I could not attain any of them. Here in Brazil the chance of getting a regular job with these traits is nil. I began then to look for freelancing work, but I could only find worthless Java/PHP/ASP/VB/etc. projects. Unfortunately, here, those are the “high-tech” jobs.

As seen in my previous post, it is possible to use Scheme to develop for the iPhone and for the iPod Touch. Developing for the App Store has the potential of fulfilling my goals. Of course it is an already crowded market, but so far it seems to be the only way out. So I paid Apple and registered as an iPhone developer. Let’s see what I can come up with.