revUp | Issue 50

My journey to Las Vegas for RunRevLive '08 was a bit of an unknown. I booked it rather impulsively and frankly with some trepidation. Most of the other developer conferences I have gone to in the past, and there have been many, were largely disappointments. I didn't know quite what to expect. This was going to be my first RevFest.

I am here to tell you that the trip was a great success on multiple levels. During one of the kitchens I drew out a book I had randomly tossed into my bag before getting on the plane. It happened to contain a disc full of very useful and complex spreadsheets that are used for calculating many aspects of options and equity pricing. Trust me, these are things that no one wants to transcode into another language. They are complex and trusted in their current form and have been for many years. In such a situation, leaving them "native" is to me the only way to fly. They also draw on math functions Rev just does not have. Sure, they could be decomposed back into Rev math limits, but that did not seem optimal or practical. They are also as it turns out "stuck" behind the wall as it were for Rev since there was no easy way to communicate with them, collecting and posting data and running calculations and getting back results. I was fully prepared, though unhappy at the thought, of spending a summer with a CS intern cranking out Visual Basic integration solutions. That is, having to build the app and interface in VB. For me, the trip to Las Vegas was going to be entirely worth the price if I could come up with a better solution.

That solution came courtesy of one Ken Ray. Ken very generously sat with me for about an hour or more helping probe what could and could not be done with Rev 2.9 and Excel integration. I say this at the risk of being a teaser since the solutions are (a) still in progress and (b) far too complex for a single article or for this one at least. Suffice it to say I was completely stoked following that little code fest. We did it, or more so, Ken did. In addition to that I finally had a chance to shake Ken's hand and thank him for all the help he had provided before, often without him knowing it. By the way that same thanks goes out for other helpers -- to Sarah Reichelt, Jacqueline Landman Gay, Mark Wieder, Jan Schenkel, Richard "Desert King" Gaskin and Andre Garzia amongst a host of others. All but Sarah were live and in person at the conference, ready to help. I would encourage everyone to think about heading out for the next RevFest. Now on to our first step into a larger world.

Revolution is a tool that I pretty much live in eight hours a day. There are many times where I need to work in other environments however to tap some special functionality or for legacy reasons. For the most part I found annoying and time consuming as I often had to write, or rewrite, the very same code in nearly all the environments I worked in but could not reuse it between them. Most of the code was "support" code for file handling, reading, writing, etc and very little of it was about doing the specific task I needed done in that language. There had to be a better way!

Up until version 2.9 of Revolution there were some very clever (and still useful) ways of having Revolution fire off scripts to other environments that Ken Ray developed and made available to the community (see Ken's site www.sonsothunder.com for details). But now, with ActiveScripting support for Windows, we have even more flexibility.

It may not be clear from the documentation that ANY language that is supported by the Windows Scripting Host is available to Revolution 2.9. What? I thought that only VBScript and JScript were supported? Au contraire mon frere, any scripting language that has an active component can be used. That, by the way, means ActivePerl, ActivePython and perhaps Lua and RubyScript are all there for the having. For me personally having VBScript alone to be able to do inter-application work, especially with Excel where I live 20% of the day, was a fantastic help. Having access to all these other languages was a completely awesome and unexpected side benefit.

Since I tend to work with numbers a lot, what I looked around for in the other scripting languages was more math functions and capabilities. Solution? Python and NumPy which will be our focus. Some of what I will mention below can apply to just about any handling of the scripting languages as there are some "gotchas" to be aware of.

Before sitting down to write this article I had exactly zero face time with Python. From a quick poke around it seemed that Python already has some great built-ins but NumPy, an add-on for Python, has even more for math and stats. Need to solve complex numbers? Imaginaries? How about matrix maths? Calculating various statistical distributions? Fast Fourier Transforms? Yes, it's all there in Python or NumPy. There are vast resources out there to learn more about the language. A few are at the end of the article.

To get ready to try this on your own, you of course have to have Python installed on your system and then NumPy. This is important. You must first make sure that you have Python installed properly on your machine BEFORE you install NumPy or it will not attach properly. For this quick and dirty test I went the easy route and installed ActivePython 2.5. Getting the version of Python and your version of NumPy in sync is also important. NumPy 1.0.4 is the correct one to use with Python 2.5. If you have Python installed and working already, more power to you. I went the ActivePython route as it does the paths set-up that you might have to do manually otherwise.

Once you have installed ActivePython and then NumPy, the next step is to test the installation. If you are the cautious type you can add an extra step and test for the Python install first. How? Easy peasy.

Regardless of what language you might be thinking of tapping into, we have to remember that not all machines will have the language installed. Further, some installs are kind of sneaky and may need to be tweaked before they are accessible. This step then applies broadly to any use of alternate languages.

On mouseUp
 put the alternatelanguages
end mouseUp

This puts the return delimited list of languages into the message box. Of course you can roll this into a function for later testing to make sure the target language is available before you go off into the weeds. Something like...

function isLangInstalled pLang
  put the alternatelanguages into temp
  if pLang is not in temp then
    return "ERROR:"&& pLang &&"is not installed."
  else 
    return empty
  end if
end isLangInstalled

You would call this in your mouseUp or other script passing "Python" as the pLang parameter to the function, or whatever your target language may be.

After this, and presuming you have installed NumPy, you can do the following to test to see if NumPy is installed. This will be the first time using Python from within Rev by the way:

function testNumpy
  put "import numpy" &CR&\
  "x=numpy.__version__" & CR & \
      "result=x"  into tScript
  do tScript as "Python.AXScript.2" –- note the do...as
  return the result
end testNumpy

A couple of things to note. The version of Python on my machine happens to be Python.AXScript.2, the Active version of Python 2.5. I have to call that whole string name or else the script does not work. The name on your target machine could be different. You may therefore want to grab and store this value somewhere when calling isLangInstalled function rather than having to call it before every instance.

Second, it takes a while for NumPy to load the first time it is called. If you actually have NumPy on your machine and it is working, what you will get back is a version number string from this function. In the event that you do not have NumPy installed you may get a fairly lengthy diagnostic string back from Python.

The result back from the function in my case was "1.0.4" which is the correct version for Python 2.5.

There is a feature that you need to be aware of when using the "do as ...alternate language" form. It's one thing to be able to execute a script in another language as we have just done in Python. But how does Rev recognize the values back? The trick is the use of the Rev "result" embedded in the script we just called.

At first I found this very confusing. After all Python doesn't know anything about the variable name "result". How can we use it? Aren't we calling another language that knows nothing about Rev's result? However that is precisely how all your values from Python (or any alternate language) will be returned to Rev. If you don't pass "result=" at some point in your script then you won't get back anything. Assignment of course can vary in languages and you may need to coerce values for Rev. I have a few examples of that below.

Presuming that Python is installed and you have your NumPy version string back, you can now think about doing something more interesting.

Anyone want to do some matrix multiplication? In Python this is a snap using NumPy:

on mouseUp
  put "from numpy import *" &CR&\
  "a = mat('1 2; 3 4')" &CR&\
  "b = mat('5 6; 7 8')" &CR&\
  "x =bmat('a b; b a')" &CR&\
  "result=str(x.tolist())"into tScript
  replace "`" with quote in tScript
  do tScript as "Python.AXScript.2"
  put the result
end mouseUp

Again a couple of things to note. First, to make use of NumPy math we have to pass the "from numpy import *" statement or that library will not be available. This is true of any additional functional libraries we may need to tap into so in the examples below you may see multiple "import" lines.

One of the "gotchas" we need to watch for in using Rev with Python is seen in how we get our result back. I noticed that unless we do some manipulation with the result in Python, we don't get back what we need, or in some cases the script will throw a cryptic "execution error" message.

To get back the result in the proper form we need to convert our matrix multiply function (from bmat line) to a Python list, by using the "variable dot tolist()" syntax. The tolist() is a built in Python function that converts arrays into Python lists. The use of the Python str() function is something that I discovered by accident through trial and error after realizing that Rev could not handle certain Python structures. We therefore have to explicitly convert the result to a string for Rev's benefit.

In Rev, after stripping away the outer square brackets, we get back four items of four elements each representing the result of the matrix operation. By the way if we do not use the conversion to list but do just a str() conversion, this is what we get back.

As far as the "gotchas" go, the above handles the most serious problems I encountered when trying to put the samples together. We have to remember to put the result back into a format that Rev is going to be able to handle and str() and "x.toList() are the keys to making this work.

What follows below are some rudimentary samples that I hope will show how relatively simple putting Rev and Python together can be. In the case of some of these functions, we would otherwise have to go to externals or do some extreme Rev tricks to get similar capabilities.

The following are simple examples take from much of the Python introductory documentation. I am including them here to show how we can extend Revs math capabilities with relative ease. You'll have to explore this further on your own. If you do, please share your findings back with the group.

This is a simple example to convert numbers to hexadecimal notation that you may find useful as a utility.

function convertToHex pNum
  put "import math" &CR&\
  "x=hex("&pNum&")" &CR&\
  "result=str(x)" into tScript
  do tScript as  "Python.AXScript.2"
  return the result
end convertToHex

I did not have time to code this up as a function but wanted to show that we can use imaginary numbers and split the real and imaginary number results in Python (in Python the "j" suffix is how imaginary numbers are indicated). In this case we are also showing the hypot() function which calculates the hypotenuse.

on mouseUp
  put "from numpy import *" &CR&\
  "z = array([2+3j, 3+4j])" &CR&\
  "x=hypot(z.real, z.imag)" &CR&\
  "result=str(x.tolist())"into tScript
  do tScript as "Python.AXScript.2"
  put the result
end mouseUp

Python let's us tap into using Long number forms as well which in Python are unbounded. This is a simple powers function (the "**" sign is exponentiation in Python) that uses Long numbers, indicated by the "L" suffix.

on mouseUp
  put "import math" &CR&\
  "result=pow(2L, 10L**100+1, 10L**100+1)" into tScript
   do tScript as  "Python.AXScript.2"
  put the result
end mouseUp

This sample was taken out of the Python introductory samples text. It shows the ease with which FFT and the inverse FFT can be called in Rev using NumPy's FFT library.

on mouseUp
  put "from numpy import *" &CR&\
  "from numpy.fft import *" &CR&\
  "signal = array([-2.,  8., -6.,  4.,  1., 0.,  3.,  5.])" &CR&\
  "fourier = fft(signal)" &CR&\
  "fourier=ifft(fourier)" &CR&\
  "result=str(fourier)"into tScript
  do tScript as "Python.AXScript.2"
  put the result
end mouseUp

Although we could code up a compound interest function in Rev, I wanted to show the ease with which we could leverage Rev and Python to produce a useful function. In the example below there is really only a single line that makes use of the special capabilities of Python,

The rest of what we are doing -- error checking and formatting the result to return the starting balance + accrued interest, the percent return and the return portion less the balance -- is all being done by Rev. I hope that this illustrates that we can have the best of both worlds and do not have to execute the code strictly in the alternate language. We can instead lever the best of both to produce a result.

on mouseUp
  put compoundInterest(100,0.045,10)
end mouseUp

function compoundInterest pBal,pRate,pYears, pFlag
  if pBal is not a number OR pRate is not a number OR pYears is not a number then
    return "ERROR: Some Params not numeric"
  end if
  put "import math" &CR&\ 
  "tBalance="& pBal &CR&\ 
  "tRate="& pRate &CR&\ 
  "tTime="& pYears &CR&\   
  "result = (1+tRate)**tTime*tBalance" into tScript 
  do tScript as  "Python.AXScript.2"
  put the result into tEval
  set the numberformat to "0.00"
  put ((tEval-pBal)/pBal)*100 into tPercentReturn
  put tEval-pBal into tReturn
  return tEval+0&"|"&tPercentReturn&"%"&"|"&tReturn
end compoundInterest

This has been a very quick trip into a larger world that Rev 2.9 opens up to us. I hope that you will all be encouraged to experiment and share your results back with the community.

Please note that I simply used the .msi installer for version 2.5.2.2 and used the free version (standard distribution)

Make sure you choose the right binary. You will want 1.0.4 for Python 2.5, which can be the .exe (numpy-1.0.4.win32-p3-py2.5.exe) or the .msi (numpy-1.0.4.win32-p3-py2.5.msi).

NumPy documentation is scattered over the map. I generally simply picked and chose sample scripts to put together the article.