Only if logging out actively invalidates session tokens, which appears to have not been the case. https://github.com/LemmyNet/lemmy/issues/3364

If you’re not familiar with it, windows now has native support to run Linux via the Windows Subsystem for Linux (WSL2). It can integrate pretty well into your environment as well, like pushing vscode applications to run in the Subsystem. It apparently supports gui as well although I haven’t tried setting that up myself.