A misadventure programming on Windows

Today, I decided to get started on learning the advance concepts of Rust programming language. I had everything I needed to get started - the books, my experience with Rust and a pleasant weather. All that and a data plan of 1GB per day.

Since my Macbook Pro is currently off sick due to a screwed up butterfly keyboard, I had no choice but to go to my Windows laptop. I downloaded the rustup installer, and upon launch was greeted with a message that I would need to install Windows build tools, which was probably going to be a download bigger than 1GB. Not wanting to go that route, I decided to use rust on WSL. The setup was easy, and within a few minutes I had rust up and running. I ran into a tiny problem, however - I needed an IDE which could work properly with Rust. Lucky for me, I already have a CLion license. But CLion will play nice only with the Rust version that is installed on its current platform, so Windows CLion plays nice with Rust on Windows, in other words, I was out of luck. So, I got to work.

Go Google

A quick Google search for Intellij wsl rust revealed that there was an open issue for the same on Github and a pull request was pending review against that issue. So someone is already working on this, huh.

A dumb idea clicks

I figured that all CLion expected was to find a bunch of Windows executables in the .cargo/bin directory on WSL. So I decided to make a wrapper executable that would accept calls for that tool and forward them to the actual tool running inside WSL, then I would forward the stdout and stderror streams of the WSL tool to my wrapper executable and pray that everything would work.

Of course there were some things to take care of, like passing of file arguments between Linux and Windows tools required path translation, and Microsoft already presents us with wslpath for that. But that can come later, I needed to work on a POC first.

Working on the idea

The simplest way to send command line arguments to a Unix executable should be something like this:

• Wrapper program masquerades as original and records all the command-line arguments. In our case, CLion probably expects the wrapper program to be an executable file.
• Some of these arguments, like filenames, need to be translated.
• Call the desired Unix program from bash -c "your_command".
• Profit.

The first problem could be solved by using AutoIt. I had been using it for a few years, and one thing I remember is that AutoIt can convert your scripts to executables.

The 2nd problem could be solved by wslpath, but I decided to defer that for later. The third problem was an immediate concern.

So, in order to call a Linux program from AutoIt, I needed to call it via bash.exe -c. I would call StdoutRead method in an infinite loop, add some exiting conditions and be done with it, except, I failed in my very first order of business, when I tried to run bash.exe and got a FileNotFound error.

Scratching… scratching

Surely enough, I had bash installed on my system. Maybe it was a PATH resolution problem, so I decided to give a full path-name: C:\Windows\system32\bash.exe. And it still didn’t work.

I wasted another 30 minutes chasing this issue, and finally gave up when I found nothing. And I decided to go try other things for a while. But where did I go wrong?

The solution

And finally, I came across this GitHub issue on WSL repo: https://github.com/microsoft/WSL/issues/1598

The author mentioned that they were trying to compile a simple C code from Atom Editor. Then I came across this reply:

Homie. It’s at C:\Windows\Sysnative\bash.exe.

It’s because bash is not in SysWOW64 (SysWindowsOnWindows64bit), i.e. 32-bit (and someone at Microsoft had a bloody good sense of humor 🙃).

@benhillis you should just symlink bash.exe in SysWOW64. This is the 8th time I’ve seen this Q on either here or StackOverflow.

But there’s no Sysnative on my machine. I scrolled further.

@Skyferia You are calling the script from a 32bit program, so bash.exe lives in C:\Windows\Sysnative. Open a 32bit command prompt (C:\Windows\SysWOW64\cmd.exe) and try it. Bash.exe will fail to launch and if you dir C:\Windows\System32, bash.exe will not be there. Trust me, just use sysnative in your script and it will work.

And so, I tried it. And it worked!

Further explanations came:

@Skyferia:

Since your 64-bit OS needs to be able to run 32-bit programs without confusing them with new paths, some paths are redirected to different folders depending on whether a 32-bit or 64-bit application looks at them. The same applies to some registry keys. Check out the articles about “Registry Redirector” and “File System Redirector” here to learn more.

Basically, if you open explorer (a 64-bit program), you will see these folders:

C:\Windows\System32 « your 64-bit system folder C:\Windows\SysWOW64 « your 32-bit system folder (Yes, the numbers appear to be exactly the other way round, for reasons too complicated to explain right now.)

However: When a 32-bit program, such as Atom, looks at your Windows folder, it will see these folders instead:

C:\Windows\System32 « your 32-bit system folder (!!!) C:\Windows\Sysnative « your 64-bit system folder

This ensures that under the path C:\Windows\System32, all applications, be it 32 or 64 bit, will find the system files with the correct bitness which fits to the applications themselves - 64-bit applications will find 64-bit files there, file 32-bit applications will find 32-bit files there.

The way to access files with the “other” bitness (32-bit from 64-bit applications and 64-bit from 32-bit applications) is to access the C:\Windows\SysWOW64 and C:\Windows\Sysnative paths, respectively.

Because bash.exe exists only as 64-bit file, it is located in the C:\Windows\System32 folder from a 64-bit application’s perspective (such as from Explorer), but in C:\Windows\Sysnative from a 32-bit appliaction’s perspective (such as from Atom).

Since C:\Windows\Sysnative is not in the path by default, you need to tell Atom to specifically look in that folder, by specifying the full path C:\Windows\Sysnative\bash.exe.

By the way, you can easily check the different perspectives by trying to explore your Windows folder from 64-bit cmd.exe (which you normally get) and also 32-bit cmd.exe which you will find in Explorer in C:\Windows\SysWOW64\cmd.exe.

If you just scrolled past this comment, directly at this point, I implore you - please read it. Read it again. You might save lives someday.

So, Windows basically shows different file system layouts for different architectural programs.

Meanwhile, from the perspective of a computer program, its raining Matrix:

Epilogue

So, after finally solving this confusing issue, I was finally able to get the wrapper working. CLion however, still didn’t play nice, so, till I can get my Macbook repaired, I’m still stuck without IDE support. Its gonna be slow, but its fine.

Under no circumstances am I ever downloading something huge like Visual Studio (don’t worry, XCode is even bigger, around 8 GB download from App Store), which I don’t need for the moment. Not to mention its installer needs to update before you can uninstall it.

Later.