Dutch log

Every day is ReSharper Appreciation Day.

ProfileTool doesn’t work on half the machines I try it on. I get an error like:

Profiletool.exe Common Language Runtime Debugging Services
Application has generated an exception that could not be handled.
Process ID=0×14 (1300), Thread Id=0xe8(232)

Click OK to terminate the application.
Click CANCEL to debug the application.

There’s no point searching Google for the process ID and thread number; they’re unique to your instance.

What can you do here? You get an error suggesting that ‘debugging failed’ if you hit Cancel - this is correct, because you don’t have a debugger installed!

First step, if you’re running your .NET application off the network, stop, move it to your local disk, and try again. The default security policy doesn’t let you do this - you can edit it in the Administrative Tools if you need to.

Now, if you still get the error, drop to a command line, and run the following:

ProfileTool 2>error.log

This redirects the standard error output to error.log, which you can then read. Linux/UNIX outputs this to the terminal by default; Windows hides it.

Thanks to Frank Racis, by way of Mohamed Yehia.

The actual error, in this case, was “No mapping between user accounts and security IDs was done“, which really means “don’t catch System.SystemException when you are throwing System.Exception”. And when you’re testing new builds, don’t sit behind an ISA server which aggressively caches the old build.

I’d like my application to be able to run on .NET 1.1, so I’ve avoided using all the new niceties of .NET 2.0.

Or so I thought.

First, I experimented with MSBee, which lets you build for the .NET 1.1 environment using the VS 2005 tools. After seeing how many errors my code generated in 1.1, while it compiled cleanly in 2.0, I thought “maybe reimplementing this in VS 2003 might be easier”.
And while I’ve specifically avoided some of the .NET 2.0 things that would have been really handy, like the SecurityIdentifier class, which makes handling SID-account mapping easy, there’s things I didn’t know would be a problem. You can’t set a registry key to be a certain type in 1.1. And you can’t use half the features that the designer implements for you automatically. And even if you could, you’d have to hack at it, because they separate code and design with partial classes in 2.0.

So, to summarize, looks like .NET was pretty crap prior to 2005. And that means you need to have XP SP2, because it’s not supported on SP1.

(/rant)

I don’t often write code, but I do find it’s a good complement to cricket watching. So, in honour of New Zealand playing Sri Lanka in the World Cup semi-final, I’m proud to announce the release of the first public beta of the IT Partners ProfileTool.

ProfileTool lets you take a Windows 2000 or XP user’s profile and assign it to a different user, without having to copy any files or perform any manual procedures.

This is a common task for people who are taking a network of PCs that have not been on a domain, and joining them to one; or if you replace a domain controller without keeping security information, which is sometimes done on sites upgrading from a badly installed SBS 2000 installation to a nicely installed SBS 2003 installation.

It also has useful features for deleting a profile off the disk/from the registry, and changing profiles from roaming to local, replicating the features of the User Profiles capplet.

The source is available under the Mozilla Public License, with the implication of “Do what you like with this code, but if you improve it, I’d like your source to come back to me please”.

Today at work I needed a Color Chooser dialog that supported Alpha. The standard Windows Color Picker from Common Controls can’t do that I don’t think so I decided to have a look at the code in Paint.Net’s color dialog. Outwardly Paint.Net looks well engineered so I went looking at the source. Crikey what a mess. Now I know how hard it is to write and manage a moderately sized code-base with a small team, but I at least try and organize my code well. I also constantly refactor to improve the design & code readability.

After spending the day trying to adapt Paint.Net’s color picker I ran into some serious performance issues (amongst other things). It seems their gradient widget renderer is very slow for anything other than the tiny little gradients that get rendered in their color picker form. I think someone needs to introduce them to LinearGradientBrush’s InterpolationColors property and ColorBlend.

Though I think the coders on Paint.Net probably already know some of their code is pretty dire, considering this comment in ColorsForm.cs.

// TODO: rewrite this … the code is out of control here as it has grown organically,
// and it’s impossible to maintain. post-3.0

Now I’m not trying to pick on the coders responsible for Paint.Net. I’d actually like to thank them, as I use the application regularly and appreciate that the code is MIT licensed. This is what enables me to go rifling through their code in the first place.

But that code is not good enough and is too Paint.Net specific so I’m going to have to write my own damn fancy color picker.

Xbaps are just multiplying by the day… Just came across this interesting xbap which runs VistaSmalltalk sessions. Its got a nice usage of popups.  

http://vistascript.net/vistascript/vsb/Vsb.xbap

Share this post

I decided to start doing more stuff with XNA. However I want to use Visual Studio Pro rather than the Visual Studio Express. Unfortunately Pro doesn’t support the XNA content pipeline or deploying to the XBOX 360. For the moment I can live without the deployment feature (my 360 is busted anyway) but I need the content pipeline. So what I’ve done is started developing an NAnt task for processing XNA Content with the XNA Content pipeline.

I’ve put the source code up on my Projects page here. So far for what I need it to do the task is sufficient, but I’m sure as my needs increase I’ll be extending it a bit.

I came across this Programmer Personality Test at Doolwind’s Game Coding Site. I would not suggest you take it.
For example:
Question 4: Who is a ‘better’ programmer?

Someone who went to university/college and has at least a masters in their field. They should read as many books on a subject as they can find before starting on it.
Someone who has been coding since they were 5 and simply goes by raw talent alone.

My answer would be neither. The options don’t provide enough information about the programmer to make a judgement. Neither extreems are desirable.
Question 5 is similarly flawed.
Which is the ‘best’ way to program?

In assembly or straight C. I’m in complete control of what the program’s going to do.
Java or any .net language. Why waste valuable time on memory management and re-inventing the wheel when someone’s handled it for you!

The “best” way to program depends on the task. If you’re worried about computer performance then C/C++/Asm might be the best option. If you’re concerned about programmer performance then maybe Java, C# or Python would be.
While tests like this can be humourous, this one is so easy to manipulate its not funny. Mainly because the “right” or “desired” answers stick out like dogs balls.
In fact this sort of test probably does more damage than anything else by promoting stereotypical and flawed views on programmer personalities.

It occured to me how much knowledge is being lost since I taught myself to program. Programmers today "know" some of the low level details of how a machine works in theory, but really, how much do they know?
When I started teaching myself to code it was on an IBM XT with DOS. Now, if you're somoene who's never really used DOS (or CP/M or…) except in "DOS Box"'s, then you really don't know what it means to run a non multitasking, non protected mode machine. DOS is not very much more than a way to load programs into memory and run them. It provides only very basic abstraction from the hardware, and most of it is extremely slow and limited. the DOS API's were generally only ever used to read/write files. DOS was essentially treated a filesystem driver.
The realmode x86 chips had a weird memory model called "segmentation". Each segment was 64k, and segments overlapped with each other offset by 16 bytes. So offset 0010 in segment 0000 was the same as offset 0000 in segment 0001. You'd write these addresses as segment:offset. Programmers would regularly directly access addresses in memory that they knew contained things. I can still remember a lot of interesting addresses off by heart.

0000:0000 was/is the interrupt vector table
0000:7c00 - the bootsector was loaded into memory
A000:0000 - the VGA Framebuffer was located.
B000:0000 - the monocrome textmode display was located
B800:0000 - the colour textmode display
C000:0000 - this was "Adaptor ROM", or where network cards (and other things) put their stuff
F000:FA6E - this was the VGA ROM BIOS Font

You talked directly to the hardware. If you wanted to hook an IRQ, go for it, overwrite the address it would use in the interrupt vector table, and remember to acknowledge the interrupt when it came time to return.
I remember to change the VGA palette you write the pallete entry you want to change to io port 0×3c8, then write the red value to 0×3c9, green to 0×3c9, then blue to 0×3c9. They could be from 0 to 63.
In fact, I remember you could poll an IO port (I must admit I've forgotten which one, 0×3CA?) and see if a horizontal or vertical retrace was in effect. Depending on the screen mode there were only 16 or 256 or whatever palette entries. Most of the time this meant that you could only have 256 colours on the screen at once. If you modified the palette while the video card was drawing on the screen you'd get "Snow".
Snow was caused when the video card tried to read a value that the host CPU was in the process of modifying. It would normally read out 0xFF or some other weird value instead so you'd get "white" in random places across the screen causing what looked like "Snow".
But you had enough time to modify about 4 entries during the horizontal blanking time (on my hardware at least) before it started painting the line and therefore avoiding the snow. This meant you could use 256*4+242 colours on the screen at once (on a 256 scanline resolution). Often this was used to change just the background colour, usually in a nice smooth gradient. This was called coppering.
By investigating what the other video registers were used for people discovered lots about how video cards worked. Video cards were only allocated 64k of address space, so to fit higher resolutions or higher bit depths in they used "planes" (or banks) that you switched into the 64k address space, wrote your updates to it, and then flipped in the next plane. Lots of interesting tricks could be pulled using these techniques. A lot of video modes were "chained". A write to video memory would write the same value to all planes simultaniously. These modes were fast and easy to program for, but they were stuck at very low resolutions. But by programming some of these low resolution modes and turning off the "chained" bit in the video card you could effectively create your own unsupported, undocumented video modes. These modes were collectively called "Mode X".
Because the machines were so slow, you spent a lot of time working on optimising code. Often you'd end up rewriting tight loops in assembly code. Even then you'd look at how you could optimise your assembly code.
x86 instructions are of dynamic length. There minimum length is 1 byte. Their maximum length was something like 6 bytes. I suspect on a modern x86_64 machine it's probably closer to 12 bytes by now.
Your bus was 8 or 16bits wide, and you had to wait for bus transactions to complete, so you'd want to choose the shortest opcode you could. Also memory was at a premium, every byte saved was going to be useful somewhere else. xor ax, ax was better than mov ax,0×0000 because xor ax,ax is a 1 byte opcode, where as mov ax,0×0000 was 3 bytes.
Unlike the RISC machines you get taught on at universities where everything is nice and simple and elegant, and instructions take a fixed number of cycles, on x86 they can take different amounts of time. xor ax,ax might take one cycle. a multiply or a divide might take a few hundred cycles. A sin or cos might take a small iceage. So sometimes you wanted to use more opcodes because they were faster but the achieved the same result.
One popular VGA video mode was mode 0×13. ("mode thirteen"). It was 320×200×256. To code a putpixel you needed to do (A000:0000 + Y*320 + X). But that had a nasty multiplication in it. Remember how you were taught to do multiplication at primary school?
32
*320
—-
0
+ 64
+96
—–
10240
Imagine doing this in binary:
0000 0000 0001 0000
*0000 0001 0010 0000
——————-
0000 0000 0000 0000
0 0000 0000 0000 000
00 0000 0000 0000 00
000 0000 0000 0000 0
0000 0000 0000 0000
0 0000 0000 0010 000
00 0000 0000 0000 00
000 0000 0000 0000 0
0000 0000 0001 0000
<snip>
———————————-
0000 0000 0001 0010 0000
The thing to note is where there is a zero in the second argument to '*', there is a line of all 0's in the working, and where there is a 1, the first number is copied entirely shifted left by the position of the bit. So to multiply by 320 you could instead do res=(y<<5)+(y<<8). This was faster, significantly so. I remember benchmarking it
Other quirks of the x86 architecture were things like you could do complex addressing modes like [bx+4*ax+3]. this was handy when you needed to find the address of a member of a structure in a array. There is an opcode called LEA. Load effective Address. This instruction takes an address as it's argument and loads that address (not the value at that address) into some other register. The fun thing was that LEA was a single cycle opcode (and quite short 1 or 2 bytes IIRC). This meant that you could do a shift by a power of two, and an addition in one opcode instead of two. And not only that it wouldn't use the ALU to do the arithmetic so another opcode could be busy using the ALU.
One of my greatest "hacks" was a program I wrote that used one of the "Mode X"'s I mentioned above to create a 256×256×256 video mode. Most monitors hated this mode as it was effectively square, and they were 4:3, but with some protesting they'd do it. I created a subdivision style "plasma". Then each vertical refresh I'd blit onto the screen a tunnel going off to infinity with this plasma textured onto it.
Originally I wrote this in Pascal (Turbo Pascal v7 for DOS was a brilliant language, compiler, and IDE). Except it was slow. Glacially slow. So I started rewriting chunks of it in assembler. But I soon ran into the problem that intel machines don't have many registers. So I stored all the registers in the code segment (writable code segments yay!), and then used all the registers (including the stack pointer — which is why I couldn't store them on the stack) as general purpose registers.
It still wasn't fast enough. One of the reasons for this was I only had a handful of segment registers. Now when you're addressing memory in x86 realmode, you give an offset into one of the segment registers (CS (code segment), DS (Data segment), ES (Extra Segment), SS (Stack Segment) and I needed to access 4 buffers (vga buffer, texture, and x/y data for my tunnel).
So, I did some research and found the opcodes for the new 386 instructions, and tried using them. But turbo pascal's internal assembler didn't know about these opcodes. Now when it comes to x86 assembler there are prefixes for changing some characteristic about the next opcode. for instance a "ES:" prefix would modify the address in the instruction to be relative to the ES (Extra segment) instead of the DS (Data segment). If you used 32bit registeres in real mode the assembler would automatically output a prefix saying to use 32bit values/registers instead of 16 bit ones for the next opcode. This assembler didn't know how to do that. So I would use "db 0×66 ; mov ax, sp" to make the instruction "mov eax, esp". But then I discovered that the 386 had introduced another couple of segment registers (gs/fs). howeve there was no prefix for them, I had to hand assemble and insert the raw hex values for them into my program.
So now I had a program that I had hand assembled to make run, and now it ran fast enough to be useful on my machine, and it looked sweet!
But around the time of the release of ID's game "Doom" all of this disappeared. Compilers didn't produce completely moronic code anymore, and processesors were fast enough that you could afford to waste a few cycles here or there. The program I'd painstakingly hand optimised above I showed to a friend. He wrote the entire thing in C and it ran fine on his pentium computer. (his version however didn't run at all reasonably on my 386).
Doom was written almost entirely in C, including the 3d raycaster. Only the dissolve between missions was written in assembler. Soon computers were coming with 3d accelerated video cards and everyone forgot how to write assembler, except for the fringe lunatics and people who write compilers.
You might think that this is progress. No more do people spend a week painstakingly hand assembling code to get things to run as fast as humanly possible. But, the skills I learnt over 10 years ago when I taught myself to program are still useful today. Knowing the layout of a bootsector means I can correctly identify a lot more problems people have with disk images. Knowing assembly language in far more detail than anyone thinks is necessary helps with debugging obscure problems in programs.
You tell people today about hand assembling programs to get them to run at speed and they just won't believe you.

But of course! It is right there!

I feel his pain.

Elephants can be so inconvenient at times.

A valid answer to a different question.

Curves are cool.

So, I’ve been doing some work on a network visualisation program called BSOD (Don’t ask), cleaning up code here, tweaking some stuff from there. Anyway I’ve got it running well under Linux. The final thing to do before a new version is released is compile it under windows.
Now, I’ve not seriously used Windows since about 1995. When Windows 95 was released my current computer more or less defined the minimum requirements, and through various events ended up using Linux. So this experience with programming under Windows was….. educational. I’m not unbiased in this; I’m used to doing things under Linux. I’ve been using Visual Studio 2005 under XP.
So, what was the experience like? Well, it wasn’t my favourite experience. Some things don’t work as I expect (Every time I try to use the middle click to paste I realise that no… I have to explicitly copy and paste it). Things are different so thats ok. What did bug me was Visual Studio. People explain to me how great Visual Studio is, but my experience was somewhat… different.
Solutions and Projects confused me. Does a solution hold projects, or do projects hold solutions? Why can’t I have multiple solutions open at once? One for building an external library and one for building the main program? Given two checkouts on the same machine, how can I tell what the path is to the file I have open in any obvious way? (I assume theres some way to find out).
Window’s weird ideas about files being open not being able to be touched is annoying. Especially when visual studio suggests that this is probably due to me using version control (wtf?).
Visual Studio recommends poor programming practises. Wizards produce bad code. One example is the skeleton code it produces when you generate a new console application:
int _tmain(int argc, _TCHAR* argv[])
This uses “_tmain”. Now in C/C++ identifiers that begin with an “_” are reserved for the standard library. Programs should avoid ever using identifiers that start with a “_” so that the standard library can use them for symbols that shouldn’t be leaked into the application. For instance on MacOS defines almost every single letter variable with a _ in front of it.
Visual studio doesn’t really try to be a C compiler. But it’s almost insulting to have the compiler tell me that my C program doesn’t conform to the ISO C++ specification. It makes me angry to have the compiler suggest changing portable functions (eg strdup()/snprintf()) to less portable versions. C99 was released 8 years ago, yet Microsoft don’t even mention <inttypes.h> / <stdint.h>. These headers are important for portability for applications that deal with integers of fixed length. Sure VS has it’s own non-portable ways of specifing these, but that just makes the oversight more annoying. The solution so far has been to implement our own <inttypes.h> and <stdint.h> for windows and ship them. Frustrating.
Linux doesn’t really have any one elegant way to describe where the library path, library name(s), and header files are for installed library. Two that spring to mind are pkgconfig, and *-config programs. However despite these rather obvious flaws, I don’t see any way to deal with this in Visual Studio eitherrather you have to add the library name(s), the header search paths, and the library search paths. And surprisingly library names aren’t selectable. Plan9’s solution is to have a “#pragma” that specifies the library to link against. Thus if you #include a header, you automatically end up linking against any library that they use.
Microsoft have said in the past that Linux is based on 30 year old technology. Under Linux things have been tried and tested. If they were found lacking then they have been replaced. Windows however appears to have managed to accumulated bizarre limitations directly inherited from the 16bit days. One example which bit me in the butt was that I can’t malloc() in my program, and then free() it in a library. Why? Because in the 16bit world malloc() allocates out of a pool of memory per DLL and a seperate one from the main program. But the limitations is annoying.
DirectX stuff appears to be complicated and rapidly changing. The code I’ve been playing with was written against DirectX 8. DirectX 8 however appears deprecated. (The documentation discusses it’s support for the new and upcoming “whistler” operating system!). However now it appears difficult to compile code against DirectX 8 now, newer SDK’s are geared more towards DirectX9 and DirectX10.
Many of the key combos I remember from my Turbo C++ days still work. The debugger works much the same way. I enjoyed playing with “Run to Cursor” again after all these years. I find using “bt full” from gdb to be much more useful than the annoying “watches”/”call stack”/”locals” interface, which appears to be more limited than I remember it being when I was using IDE’s in DOS.
I miss the flexibility I feel I lose moving to VS. VS seems to want things done in it’s preferred way. Sure you can override it, but you’re swimming against the current. I miss regular expressions, grep, and sed. For example “Find all references” doesn’t show any context, how do I know which one is the one I want? Why can’t I compare two files to see how they differ like I can with diff(1)?
I feel like I’m in Beijing and I don’t speak the language.

If you need a Syntax Highlighting Code/Text editor widget for your C# projects you might come across ScintillaNET. I commend the guys involved with that project for putting in the effort but after reviewing their output I decided that embedding Scintilla wasn’t a great solution for my application.
Instead I’ve integrated the ICSharpCode.TextEditor widget from SharpDevelop 2.2. SharpDevelop is licensed under the LGPL and as the TextEditor widget is a DLL this is compatible with both Open Source and proprietary software.
The SharpDevelop TextEditor widget is fantastic and supports an absolute cavalcade of awesome features such as syntax highlighting, text folding, code completion, and more. It is also amazingly easy to use in a C# or VB.Net application. I highly recommend using it if you have the need.
The only thing I could ask for is for it to be distributed as an independent control outside of the greater SharpDevelop project.

Windows.Forms has a simple TreeView control built around the standard windows Common Control. It supports inline Label editing by setting the LabelEdit property to true. This causes the TreeView control to respond to mouse clicks and present an edit field. However there doesn’t appear to be an easy way of initiating label edits via alternate methods. For example by pressing F2 or using a context menu.

What to do? Use the Win32 API.

The following code snippet triggers treeview label editing.

namespace TreeViewExample {
public sealed class TreeViewMessages {
public const uint TV_FIRST = 0×1100;
public const uint TVM_EDITLABELA = (TV_FIRST + 14);
public const uint TVM_EDITLABELW = (TV_FIRST + 65);
}

public class Win32 {
[DllImport(”user32.dll”, CharSet = CharSet.Auto, SetLastError = false)]
private static extern IntPtr SendMessage(
HandleRef hWnd, uint Msg, IntPtr wParam, HandleRef lParam);
}

public class TreeViewUtils {
public static void StartLabelEdit(TreeView treeView, TreeNode node) {
IntPtr result = Win32.SendMessage(
new HandleRef(treeView, treeView.Handle),
TreeViewMessages.TVM_EDITLABELA,
IntPtr.Zero,
new HandleRef(node, node.Handle));

if (result == IntPtr.Zero)
{
throw new Exception(”Failed to send EDITLABEL message to TreeView control.”);
}
}
}
}

So as you may remember, I've been thinking about (and quietly implementing) a distributed filesystem.  Tonight I got as far as it mostly working, and since everyone around here is sick of me telling them about it I thought I'd tell you all about it instead
It uses the 9p protocol to talk to the kernel (although I'm using the 9p python client for testing).  I use the chimera DHT for indexing, and libnpfs for dealing with the low level details of the 9p protocol.  I've managed to find bugs in all of these pieces of software so far
 Anyway, onto the interesting part of the show:
9p> ls
foo/
9p> cd foo
9p> ls
readme
9p> cat readme
Hello World
9p> mkdir sample
9p> ls -l
-rw-r–r– perry perry 0 readme
drwxr-xr-x perry perry 0 sample
9p> cd sample
9p> ls -l
9p> put testfile
9p> ls -l
-rw-r–r– perry perry 0 testfile
9p> cat testfile
This file is a test
9p> 
You may notice that the file sizes are all 0, even though most of them seem to contain data, this is coz I've not finished implementing stat(2) yet.  Permissions etc are kinda implemented (they're stored, but not read off disk yet, although libnpfs appears to verify them for me).
There's a bit of polishing off to do before the "local" part of the filesystem is mostly viable. Then I just have to hook all the distributedness together and I should have a fully functional prototype! (yeah right…)

there is a python module called ctypes. This module lets you make calls to various functions inside a dynamically linked library without having to explicitly write wrapper code for it.  But you must specify the complete type of the function you're calling before you call it, so that the ctypes library knows arguments to push onto the stack and what order etc.
This always seemed to be a silly restriction, in an strong, but dynamically typed language such as python having to forward declare things seems to defeat the entire purpose.  And besides, when you compile something, you can compile it with full debugging symbols, which has the full type information about everything in your program anyway.
So, you could write a program to parse the debugging symbols out of an .so, and generate a python module with all the various type definitions and functions exported from the library, and you could do all of this on the fly.
And thus, I wrote it.
foo.c
struct function2_return_t {
int a;
int b;
};

struct function2_return_t function2(int a)
{
struct function2_return_t ret;
ret.a=a;
ret.b=a+1;

return ret;
}
And the example usage:
>>> import ct
>>> ct.load("foo.so")
>>> import foo
>>> ret=foo.function2(1)
>>> print ret
<ct.function2_return_t object at 0×2aefd8f12650>
>>> print ret.a
1
>>> print ret.b
2
It's all still very prototypeish, it doesn't deal with pointers or strings yet. As you can see it deals with structs (as if they were classes), and could fairly easily deal with arrays and pointers to structs (although once you get into pointers, you get into "who owns this memory", perhaps requiring a .free() method on classes).  Strings should also be fairly straight forward if you can figure out who actually owns the memory (should it be freed?).
At the moment I run and parse the output of readelf to get the debugging information, and I don't properly hook the "import" mechanism to make it all completely transparent, but as a prototype I thought it was pretty cool. 

So if I want to migrate a networking process, chances are it's going to migrated to a machine that nolonger has the IP address that it started with, which breaks all current TCP/IP connections.  Suck.
But with IPv6, we have an abundance of addresses (264 per subnet), so we could trivially aquire a new IPv6 Address per process.  Mobile IPv6 allows you to migrate the IP address along with the process.  The kernel could do the IPv6 allocation, and route all packets to that address to the userspace process, and make sure that any packet arriving from userspace has the right source address.
Since all the networking state is stored in a library in userspace, when you serialise/hibernate the process (and restore it again), all your networking state is immediately accessible and usable.  This also fits well with Van Jacobson's NetChannels allowing for perhaps some of the performance increases he has seen.

US game development industry lifestyle drives Insomniac developers east, far east. Kotaku us reporting that some US developers are so feed up with the constant crunch that they are fleeing to the east for better working conditions. Just don’t tell the Chinese coal miners and slave labourers.