Sikuli – a scripting language based on screenshots

There is a pretty cool, new scripting language, Sikuli which is a “research project developed by User Interface Design Group, MIT Computer Science and Artificial Intelligence Laboratory (CSAIL)“. The cool thing is that you can automate GUI interactions by using screenshots of buttons, sliders, input boxes, etc. But rather before I start to explain how it works, just check out the video.

The language itself is based on Jython and thus you can use Jython to do almost everything you like and it is fairly simple to use:

Example: Sikuli Code

So now you finally can automate your optimal FarmVille production schedule by having Sikuli interact with the applet ;-). Just put your production schedule into Sikuli and let it do the rest. Here is a GMPL model (based on the AMPL model from O.R. by the Beach – I just changed “;” and removed the “option” lines) for determining the optimal production schedule that maximizes cash. Can be solved to optimality with GLPK in 0.1 secs – the Sikuli code is left as an exercise 😉

param T;
set Horizon := 0..T;
set Crops;

param TotalArea;
param InitialArea;
param InitialMoney;
param PlowCost;
param Growth{Crops};
param Cost{Crops};
param Revenue{Crops};

var Plant{Crops,Horizon} integer >= 0;
var Area{Horizon} >= 0, <= TotalArea;
var Money{Horizon} >= 0;

maximize z: Money[T] + 4*PlowCost;

subject to

area0: Area[0] = InitialArea + sum {i in Crops} Plant[i,0];

area{t in 1..T}:  Area[t] = Area[t-1]
      + sum {i in Crops} Plant[i,t]
      - sum {i in Crops : t >= Growth[i]} Plant[i,t-Growth[i]];

money0: Money[0] = InitialMoney - sum {i in Crops} (PlowCost + Cost[i])*Plant[i,0];

money{t in 1..T}: Money[t] = Money[t-1]
      - sum {i in Crops} (PlowCost + Cost[i])*Plant[i,t]
      + sum {i in Crops : t >= Growth[i]} Revenue[i]*Plant[i,t-Growth[i]];

solve;

display z;
display {i in Crops, t in Horizon : Plant[i,t] > 0} Plant[i,t];
display Area,Money;

data;

param T := 36;
set Crops := SB EP WH SY;
param TotalArea := 144;
param InitialArea := 0;
param InitialMoney := 323;
param PlowCost := 15;

param:

Growth   Cost   Revenue :=
SB      1      10       35
EP     12      25       88
WH     18      35      115
SY      6      15       63;

end;