My Name Rhymes

Visualizing Permutations

2010-04-29T21:10:00Z

At some point a few years ago, I got interested in permutation algorithms and implemented a few that were in Knuth as well as gathered some that were floating around the internet. I never did anything with them, until I saw Aldo Cortesi's excellent sorting visualizations which reminded me of a figure from Knuth, and inspired me to create some visualizations of my own using Aldo's code.

Lexicographic Permutations

Informally, the permutations of a set are all possible orderings of its members. The permutations of the set {1,2,3}, are:

[1,2,3], [1,3,2], [2,1,3], [2,3,1], [3,1,2], [3,2,1]

This particular ordering of permutations is in lexicographic order; it's listed in the same order as it would be in the dictionary. There are many well-known algorithms for generating permutations in lexicographic order. Here is my version of one such algorithm; go read Knuth if you're at all interested in learning more.

Here's what a lexicographic permutation of four elements looks like:

It's easy to see that each element gets its turn at the top of the list, and that each time a new element goes to the top the remainder of the list is sorted.

Single-transposition Permutations

It is an interesting and non-obvious fact that there's a way to permute any given set by only switching the positions of one pair of elements per iteration. This permutation is deeply related to the Gray code, which if you haven't heard of, I highly recommend you go read about. The Knuth paper I mentioned aleady has a superb bit on the Gray code.

This image demonstrates clearly that at each step, there is exactly one crossing. My implementation of this algorithm is also on github.

CLP Permutation

I know very little about this algorithm, except that I got it from a message to comp.lang.python where it's attributed to Alex Martelli, Duncan Smith, and somebody named Anton (Anton Vredegoor?). Despite the crazy number of switches, and the fact that it reorders the list it's passed, it's actually crazy fast.

I'd love to hear from anyone with more info on this algorithm; my slightly modified version is here.

Odds and Ends

Well, that's it, just wanted to post some fun pictures of permutations, I hope you enjoyed it. The code I used to generate the pictures is derived from Aldo Cortesi's wonderful sortvis, and all my modifications to it are available on github as well.

If you want bonus points, I never got around to implementing Knuth's algorithm E (it's given towards the end of this), and I'd love for somebody else to do my work for me. If you're tough enough, that is.

Evaluating A Few Bracket Picking Models

2010-03-24T01:00:00Z

Since I made the bracket randomizer (please go read that if you haven't, or this won't make any sense to you) I've been wondering about its performance.

Included in the bracket randomizer are four models used to pick which team will win a game; one is quite simple and the other three are variations on a slightly more complicated scheme:

The Pomeroy Strict Model: simply choose the higher ranked team
The Pomeroy Random Model, n reps: compute the win probability of each team in a game. Choose a random number n times; if it's less than the underdog's win probability each of those times, the underdog wins. Else the favorite does. n is 1 for "lots of randomness", 2 for "some randomness", and 3 for "not much randomness".

For fun, I hacked up a couple other possible models to compare to the ones above:

The Seed Model: choose the better-seeded team until the final four, then pick randomly
The Pure Random Model: always choose randomly

To check how well the models predict, I ran each one 10,000 times and calculated how many games it predicted correctly:

We can see immediately that, of 38 games so far, the higher-seeded team has won 30 of them and the higher Pomeroy-ranked team has won 33. It should also come as no surprise that the random model most frequently predicted half the games right.

As the value of n in the Pomeroy random model increases, the curve moves from halfway in between the random and the Pomeroy strict model, with a large spread of possibilities, to a closer and closer approximation of the Pomeroy strict model.

In most bracket pools, the number of games doesn't matter as much as the number of points scored. It should not surprise us that this graph doesn't look much different from the graph of total games predicted accurately.

The most important statistic for your bracket pool is the number of points you can still win; with Kansas (a #1 seed) out already, many people have lost their predicted champions. This creates an immediately obvious pattern in the possible remaining points for the seed model.

Since the seed model always picks all four #1 seeds to the final four, then chooses randomly among them, half the time it picks Kansas to lose its semifinal, and has a pretty darn good total points remaining score. Half of the remaining time, it chooses Kansas to win the national championship and has a poor one; otherwise Kansas loses in the championship and the remaining score is in the middle. This explains the seemingly strange shape at the top of the graph.

Another thing the whole graph shows us is the true awfulness of the pure random model; its most common result leaves your bracket with an underwhelming 37 points remaining.

Let's zoom in and check out what's going on down below the seed model.

Again, as n increases in the Pomeroy random model, it approximates the strict Pomeroy model more closely. What's interesting to me, though, is between the two and the three repetition models; the two repetition model has a much greater spread than the three, and many fewer cases right near the Pomeroy line. I don't have a good theory for why this is, so if you do, please let me know.

In my bracket pools, anybody with a possible score over 100 is doing well, and anybody with a possible score of 140 or higher is kicking serious butt. If you picked your bracket using the Pomeroy strict 2 rep model (like I did, incidentally), you have roughly a 16.3% chance of having >=140 points remaining, and a 71.4% chance of having >=100 points remaining.

Unfortunately, I don't have good data to know what is a good "points remaining" score. If you do have access to such data, please let me know.

But what if you want more? What if you want not just to win your office pool, but to beat everybody in the bracket? Well then, you'll want to zoom in on the very high end of the distribution.

The Pomeroy model with the most randomness has the very highest scores, with two brackets having a whopping 177 possible points remaining. It's also apparent that the less random a model is, the less highly unlikely events it creates, and the less likely it is to produce a world-beating near-perfect bracket.

If you've got an idea for a different model you'd like to try out, or you just want to show me where the bugs in my code are, you can go check it out at github.

The Bill Mill NCAA Bracket Randomizer, 2010 edition

2010-03-14T20:15:00Z

I've updated my NCAA bracket randomizer for the 2010 field; check it out here. As of now, there are no improvements, but those may come in the next few days; feel free to suggest ideas.

Multi-Line Lambdas in Python Using the With Statement

2009-08-20T22:55:00Z

Python does not have multi-line lambdas because Guido dislikes them aesthetically. However, with just a bit of introspection, code like this is possible:

>>> with each([12, 14, 16]):
...   def _(x):
...     print x
...     print x+1
...  
12
13
14
15
16
17

I'll say a bit about my motivation for creating code like this, show how easy it is to write, and then I'll argue that code like this is both pythonic and aesthetically appealing in some circumstances.

A Mystery Solved (By Holmes Himself!)

When I first saw code using the with statement, my hope was that it would be able to be used somewhat like Haskell's Where clause or Ruby's blocks. When I dug into the spec, I was disappointed to discover that if it was possible, it wasn't easy, and I pushed the thought aside.

That was a couple years ago, and I didn't give it a moment's thought until I saw a blog post by Richard Jones that uses a with statement in exactly the way I had considered impossible up to now. I spent a few hours trying to figure it out, but I was stumped, so I put up a question on Stack Overflow to see if somebody could show me how he did it.

Within a few hours, Alex Martelli himself chimed in with a wonderful solution. The gist of the answer is that you can use the inspect module to access the context manager's calling scope, and figure out what variables have been defined between its __enter__ and __exit__ functions. I'm glad I asked aloud, because even if I had stumbled close to the solution, I surely wouldn't have come up with one as complete as his.

The How

Once I had Alex's proof of concept code in hand, I went to work making it do what I'd had in my head so long ago. In about an hour, I was able to write code that looks like this:

@accepts_block
def each(iterable, block):
  for i in iterable:
      block(i)

with each(["twelve", "fourteen", "sixteen"]):
  def _(x):
    print x

@accepts_block
def bmap(arr, block):
  return map(block, arr)

with bmap([1,2,3]) as foo:
  def _(x):
    return (float(x) + 1) / 2

print foo # [1.0, 1.5, 2.0]

What you see above are two functions which use a decorator giving them access to the function defined within the with block. The decorator passes the block to the function as its last argument just like in Ruby.

To understand how this happens, you need to know how context managers work. Context managers consist of a class with __enter__ and __exit__ methods which are called upon entering the with block and upon exiting, just as you'd expect.

Alex's solution involves scanning the scope of the calling function from the __enter__ and __exit__ methods, and pulling out the differences between them. These differences will be all the variables that were defined in the with block. A sketch:

class FindInteresting(object):
  def __enter__(self):
    f = inspect.currentframe(1)
    self.already_defined = dict(f.f_locals)

  def __exit__(self):
    f = inspect.currentframe(1)
    #pick out the differences between f.f_locals and self.already_defined

When we pick out the differences between the two, we need to be careful to check for names that have been redefined so that we don't miss out on new functions that reuse old names.

def __exit__(self):
  f = inspect.currentframe(1)
  interesting = {}
  for n in f.f_locals:
    newf = f.f_locals[n]
    if n not in self.already_defined:
      interesting[n] = newf
      continue
    anf = self.already_defined[n]
    if id(newf) != id(anf):
      interesting[n] = newf

After this function has run, interesting is a dictionary which (probably) contains all the names and values of the variables that have been redefined in the with block.

Because we have to use the id check to determine if a name has been redefined, and Python sometimes caches objects in memory, our function can be fooled. In this case, interesting will not detect x because it's being redefined and cpython caches the low integers, so id(x) will be the same for both xs.

x = 1
with FindInteresting:
  x = 1

In general, the cpython runtime is not aggressive about caching, but you should know that this possibility exists. If you use this technique, I recommend being strict about checking only newly defined functions, since there's no way to be sure if you missed any redefined names.

To make the teaser code at the top of the article work, I just wrapped Alex's code into a decorator that returned a context manager, then called the function being decorated with the definitions that we found in the interesting dictionary. The context manager's __call__ function gets overridden to allow you to pass in arguments for the function being decorated.

def accepts_block(f):
  class BlockContextManager(object):
    def __call__(self, *args, **kwargs):
      self.thefunction = functools.partial(f, *args, **kwargs)
      return self

    def __enter__(self):
      #do Alex's magic, just as above
    
    def __exit__(self):
      #make the interesting dictionary, just as above

      if len(interesting) == 1:
        block = list(interesting.itervalues())[0]
        assert isinstance(block, type(lambda:None))
        self.thefunction(block)

  return BlockContextManager()

It looks complicated and nested, but all it's doing is saving the function and all its arguments, grabbing the definitions from the with block, making sure there's only one definition and it's a function, then tacking it onto the end of the arguments list for the function and calling it. Phew.

The code above handles the case where you don't need to store the result of the function being decorated:

@accepts_block
def each(iterable, block):
  for i in iterable:
    block(i)

But what if we want to store the result? Turns out, we can further abuse the with block by hijacking its as clause. Because a variable defined in the as clause gets detected by Alex's code, we can use the inspect module to change that variable so that after the with block it reflects the result of our computation.

First we check to see if we probably have a block and a variable in the as statement, then we reach in and store our result there if we are in that case:

def __exit__(self):
  #exactly as before; frame = inspect.currentframe(1)

  if len(interesting) == 1:
    #exactly the same as before
  elif len(interesting) == 2:
    block = None
    savename = None
    for n,v in interesting.iteritems():
      if isinstance(v, type(lambda:None)): block = v
      else: savename = n

    assert savename and isinstance(block, type(lambda:None))

    frame.f_locals[savename] = self.thefunction(block)

Which lets us do this:

@accepts_block
def bmap(iterable, block):
  return map(block, iterable)
  
with bmap([1,2,3]) as result:
  def _(x):
    return x**2

print result #[1,4,9]

This time, we're really taking a leap by assuming that if we find a callable and any other variable, that the variable is where we want to store our results. This can lead to somewhat unexpected results:

with bmap([1,2,3]):
  not_a_result = 12
  def _(x):
    return x**2

print not_a_result # [1,4,9] instead of 12

That's my extremely long-winded description of how to abuse the with operator. If you want to see the full function and the super-lame test code I wrote, you can head on over to github and check it out.

Aesthetics

It should be clear from all of the disclaimers I've had to put into this article that this technique is of limited use in Python as it stands today. I'd like to make an argument that it suggests some nice syntactic sugar for python to support someday, while remaining totally ignorant of the actual difficulties of putting it into the language.

To do so, I'll start by posting the motivating example for decorators from the relevant PEP. It argues that this code:

def foo(cls):
  pass
foo = synchronized(lock)(foo)
foo = classmethod(foo)

is not nearly as readable as this code:

@classmethod
@synchronized(lock)
def foo(cls):
  pass

The main problem with the readability of the first snippet is not that it requires 2 redefinitions and 4 repetitions of foo. Rather, the main problem is that it places the cart before the horse by putting the function body ahead of the declarations that are required to understand it.

Similarly, when we define callback functions before we use them, we're required to define the function body before the place where it will be actually used. Often, we see:

def handle_click(self):
  foo = self.flim()
  if foo:
    self.flam(foo)

onClick(handle_click)

When it would be clearer to write:

with onClick():
  def _(self):
    foo = self.flim()
    if foo:
      self.flam()

Which I find much more appealing.

Conclusion

I expect that there's no way that syntax like this could be officially supported by Python, both because of syntactic constraints and the BDFL's aesthetic concerns. I do think it is a neat exercise in pushing the Python interpreter and syntax past where they want to go, and I hope that it gives some food for thought on an interesting Python syntax.

I'm excited to see where Richard Jones goes with his project, and the design choices that he makes in it, since he's pushing the boundaries of Python design. Many thanks to him and Alex Martelli for sending me down quite an enjoyable path.

Finally, in case you missed it above, go ahead and take a look at the code on github.

If you want to leave a comment, I suggest leaving it on reddit.

Update:

Someone has taken this technique a bit farther, using some bytecode hackery.

What Gladwell is Missing: Institutional Memory

2009-05-31T16:50:00Z

Malcolm Gladwell's recent article, ostensibly about how Davids can beat Goliaths, featured great praise for the full-court press as a method for less-skilled teams to beat more-skilled ones. He writes:

In the world of basketball, there is one story after another like this about legendary games where David used the full-court press to beat Goliath. Yet the puzzle of the press is that it has never become popular. People look at upsets like Fordham over UMass and call them flukes. Basketball sages point out that the press can be beaten by a well-coached team with adept ball handlers and astute passers—and that is true... Playing insurgent basketball did not guarantee victory. It was simply the best chance an underdog had of beating Goliath.

After his piece was rightly criticized all over the web, ESPN published a long and interesting email conversation between him and Bill Simmons that addressed the criticism. In it, Gladwell defended his column:

After my piece ran in The New Yorker, one of the most common responses I got was people saying, well, the reason more people don't use the press is that it can be beaten with a well-coached team and a good point guard. That is (A) absolutely true and (B) beside the point. The press doesn't guarantee victory. It simply represents the underdog's best chance of victory… I went to see a Lakers-Warriors game earlier this season, and it was abundantly clear after five minutes that the Warriors' chances of winning were, oh, no better than 10 percent. Why wouldn't you have a special squad of trained pressers come in for five minutes a half and press Kobe and Fisher?… Best case is that you rattle the Lakers and force a half-dozen extra turnovers that turn out to be crucial. And if you lose, so what? You were going to lose anyway.

Although lots of people have responded to this rebuttal, I haven't seen anyone mention what I consider to be an important reason that many teams have chosen not to implement full-court presses: Institutional Memory.

Let's imagine that the Warriors had in fact, put on the press, and that it had worked. The Warriors won against the vaunted Lakers! How would the team respond? They would press more. When that worked to win some more games, they would probably trade some of their players who were less well suited to the press to make room for some younger, faster, fitter players. They would win a few more games than they had been before. Everything's going good, right?

Then they hit a snag; No team has won the NBA Championship by pressing . The Warriors could optimize like crazy for the press, but they'd be training and working and trading and drafting to be, at best, a pretty good team. But no NBA team wants to be a pretty good team—they all want to win championships.

The mental exercise reveals what Gladwell has missed: the Warriors aren't playing to win that game, or even the most games that season. They're playing to try and win an NBA championship, and to do so, they need to spend years trying to build up the institutional memory of how to win games in the traditional way so that they can eventually beat every team, not just to win more games than they did playing traditionally.

All of this leads to a simple admonishment when analyzing organizations: don't expect that the organization is optimizing for what you think they're optimizing for. And when an organization seems to be acting in ways that are surprising to you, look for metrics that may be more important to them than the ones you expect.