Did you install the steam version? If so you need the fixed exe because the steam installed exe has problems going into windowed mode. http://www.mediafire.com/?c5b57xv25a7qq2j replace this exe with the one in your gta3 folder.

Another thing for me is that with DxWnd, I cannot have my controller plugged in, otherwise the game crashes on startup.

I've had massive troubles with dxwnd and the steam version. It worked on my laptop, not on my PC. Never found a fix, had to get a non-steam version for it to work. The only thing i can suggest is when it did work on my laptop, i had to launch the game through steam, not via dxwnd.