Custom action DLL for visual studio

Create New simple dll, the dll function signature should be:

EXTERN_C __declspec(dllexport) UINT __stdcall testsystem(MSIHANDLE hInstall)

You must follow this exactly  – beside the function name that can be anything you want.

You can have more then one function in the same dll, and call them for different actions.

The function name is your entry point (last picture).

You also must insert msi.lib to your project.

Now, returning from this function does the trick:

if you return ERROR_SUCCESS, then everything is ok, and setup commence. if you return ERROR_INSTALL_FAILURE however, setup will stop (error codes).

if you want to display a message box, make sure you get the installer window before you do that! like this:

HWND main=GetForegroundWindow();
MessageBox(main,L"You can't install this application, your computer SUCK!",L"Error",MB_OK+MB_ICONERROR);

In this case, you return an error after and do not proceed with the functions.

most cases you don’t want this dll in the installation folder (or anywhere on the destination machine) to achieve this, just mark the exclude to true.

What else? conditional action? This is really easy. You should add a custom dialog in the user interface section

And in its checkbox property (you can have more then one in more them one dialog)

set the value to something you can then relate to in the custom action editor.

If you put a condition, it will work only if the condition is met.


if you want to test your dll with a custom exe (and you should) the function signature is not testsystemas you might expect. It’s actually _testsystem@4, believe it or not! when doing a __stdcall calling, it will mangle the name. BTW  – this is fine for Visual studio, so don’t go and ruin this with a .def file.