X86 Assembler inlined in Smalltalk
Return to home page
Comments Loading...
2007-11-18

Recently I posted on how you can execute arbitrary assembler code from Smalltalk without creating external DLLs or special primitive plugins. Continuing on that, I've implemented a full X86 assembler based off NASMs command set.

Why? Well, basically because I want to play with MMX commands, do fast array transformations and let people like David Buck make ultra fast alpha blending for the X86 platform without having to create a VM plugin.

There's so much to talk about here.. so I'll point you to the packages and let the comments speak for themselves. Load up Assembler and Assembler-Tests from public store. Below is the class comment for MachineCodeX86 - the main class for the library. Take a look at Assembler-Tests for lots of examples, including a working ARC4 encryption library.


MachineCodeX86 is a concrete implementation of a machine to create X86 assembly code on.

You can use this class in two ways:
a) you can instantiate an instance and use its register variables to build up machine code in the @stream variable and then use those bytes in any way that you see fit, or
b) you can make a subclass of this class much like you do with ExternalInterface and put methods on the class that will compile in to assembler code that can be called from Smalltalk code


Using MachineCodeX86 for scripting
----------------------------------------------------

This is the long hand way of writing assembly code, since you always include a receiver with every command.
asm := Assembler.MachineCodeX86 new.

Once you have an assembler, you can access the registers and send commands to them, eg:
asm eax mov: 1

As you send the commands, the @stream will build up containing the X86 assembler bytes you can use.
You can use memory addresses in your assembler code with the #m method, eg:
asm eax m mov: 1

You can do this the more classic "assembler" way by first wrapping your call in an #assemble: call, eg:
asm assembe: [
[asm eax] mov: 1
].

The above #assemble: wrapping is not required if you subclass MachineCodeX86 and make methods directly.
Once you are finished, you simply send:
asm stream

This will return you the stream of bytes. You can now apply them using an ExternalProcedure and call them, however you must provide type information, eg:
AssemblerProcedure new
initializer: asm stream contents;
type: (CProcedureType resultType: External.CVoidType void argumentTypes: #() argumentNames: #());
owner: ExternalInterface;
processor: 'i386';
yourself


Using MachineCodeX86 subclasses
----------------------------------------------------

This is the prefered way of building X86 assembler, as it makes writing the code much easier. Make a subclass of MachineCodeX86 as you would with an ExternalInterface. Now you can define new methods and apply the <asm: > pragma to declare that this method should build an AssemblerProcedure when compiled, eg:

exampleMethod
<asm: void Example(void)>

This example will install #Example as a shared variable on the subclass you have made containing the assembler code with no return value and no parameters (and no assembler for that matter). You can specify the place to install the shared variables by using #target, eg:

target
^self class environment

You can use the same C declaration commands with the <asm: > pragma as you do with ExternalInterface C types, eg:

exampleMethod
<asm: unsigned int Example(void)>
eax mov: 1

You can now call this code and the result should be 1, eg:
MyClass.Example call

Since this is only an X86 assembler, we may want optimize some piece of Smalltalk code on X86 but then fallback to the slower Smalltalk code if we're not on X86. We can do this with the <fallback: > pragma, eg:

exampleMethod
<asm: unsigned int Example(void)>
<fallback: [1]>
eax mov: 1

Note: the fallback: parameter must be a block. This is important when you have parameters, eg:

add: a with: b
<asm: unsigned int Add(unsigned int a, unsigned int b)>
<fallback: [:a :b | a + b]>
eax mov: a.
ebx mov: b

In the above example, the parameters @a and @b are parsed in at compile time as memory addresses. By doing 'eax mov: a' we fill the eax register with the value of the parameter. If this were the address of an array, we can get the address of the array by dereferencing the variable, eg:

accessAnArray: array
<asm: void Example(void* array)>
eax mov: a "pointers to the position on the stack, so eax is filled with the address of the array"
eax mov: [eax] "now eax will point to the array"

There is one more pragma you can use, which is <raw> that will remove the VMs safety exception catching code when you call the assembler code. This is purely to do raw access and may increase speed slightly, though make execution slightly less safe, eg:

exampleMethod
<asm: void Example(void)>
<raw>
[12345] mov: 1 "this will probably crash the VM because we're in raw mode"

You can do labels and jump to them using two different label commands. The first is #label, which places a label jump point immediately on call, eg:

infiniteLoop
<asm: void InfiniteLoop(void)>
| start |
start := self label.
start jmp

The other is a future label that can be placed at some future point in the program and jumped too

skippingJump
<asm: unsigned int SkippingJump(void)>
| end |
end := self futureLabel.
eax xor: eax.
end jmp.
eax inc.
end plant

You #plant the future label where you want it to actually be and past references to it will be updated. Future labels will always use a dword jmp so that there's space to fill in the command if the jmp ends up being far.