A tiny encoding bug that exposes a fundamental architectural difference
Open the same file in VSCode. Save it. Open it in Visual Studio. The characters are different.
No one touched the encoding settings. No conversion was triggered manually. Yet one tool preserved the original characters perfectly, and the other silently replaced them with ?
This isn't a bug report. It's a story about a design trade-off — and what happens when two tools make completely opposite choices.
The Setup
I was working with a large SQL file that contained lines like this:
'No Injury — patient had no injuries...'
'Minor — resulted in application of bandage...'
'Moderate — resulted in suturing...'
The file had been created years ago on Windows, saved with Windows-1252 encoding — the classic Microsoft character encoding. Everything looked fine in Visual Studio. But every time I edited the file in VSCode and saved it, those em-dashes (—) silently turned into ?
Five characters. No warning. No error message.
To make it stranger: Git showed no changes at all.
Two Tools, Two Philosophies
To understand why this happens, you need to look at what these two editors are built on — because they couldn't be more different.
Visual Studio is a native Windows application written in C++. It has been a Windows-only tool for its entire existence. When it reads a file, it uses the Windows API directly:
ReadFile() → IMultiLanguage2 (COM interface)
IMultiLanguage2 is a component that Microsoft built specifically to handle encoding detection on Windows. It knows Windows-1252 intimately. It knows the patterns. It was designed by the same company that invented the encoding. It almost always gets it right.
VSCode, on the other hand, is built on Electron — a framework that lets web technologies run as desktop apps. Under the hood, it's Node.js. When it reads a file, it uses:
fs.readFile(path) // raw bytes
Then it has to figure out the encoding. For that, VSCode uses a JavaScript library called jschardet — a port of Python's chardet — which analyzes byte patterns and makes an educated guess.
The key word is guess.
Why the Guess Goes Wrong
Our SQL file was almost entirely plain ASCII — English letters, numbers, SQL keywords. Only 5 bytes out of 626,000 were non-ASCII: the em-dashes, each stored as byte 0x97 in Windows-1252.
jschardet looks at the file and sees 99.9% ASCII with a tiny handful of ambiguous bytes. It can't confidently say "this is Windows-1252," so it falls back to the modern default: UTF-8.
Now VSCode has a problem. Byte 0x97 is not valid in UTF-8. It's an incomplete sequence. So when the file is saved, VSCode does the responsible thing for a UTF-8 file — it replaces the invalid byte with the Unicode Replacement Character U+FFFD:
Before (Windows-1252): 0x97
After (UTF-8): 0xEF 0xBF 0xBD → "?"
One byte becomes three. The em-dash is gone. And the file is now technically valid UTF-8 — which is why VSCode never complains.
The Cross-Platform Trade-Off
Here's the irony at the heart of this story.
VSCode is used by millions of developers on Mac, Linux, and Windows. This is one of its greatest strengths — one tool, every platform. But to achieve that, it can't rely on Windows-specific APIs. It needs cross-platform solutions. So it uses Electron, Node.js, and JavaScript libraries like jschardet.
Visual Studio only runs on Windows. That's a limitation — but it's also a freedom. It can go deep into Windows internals, use COM interfaces, and rely on Microsoft's own encoding detection that was built for exactly this purpose.
The cross-platform design that makes VSCode so popular is the same design decision that caused this bug.
This isn't a criticism of VSCode. It's an honest look at trade-offs. Every architectural choice comes with a cost. VSCode chose reach over depth. Visual Studio chose depth over reach. Both choices are valid — but they produce different behavior on the same file.
Why Git Was Silent
The corruption had already been committed to the repository before anyone noticed. Git stored the ? version. The working file also had ? So git diff showed nothing — because technically, nothing had changed since the last commit.
The damage was invisible precisely because it had been there long enough to become the new baseline.
The Fix Is Simple — Once You Know the Cause
In VSCode, click the encoding indicator in the bottom-right status bar and select "Reopen with Encoding → Windows 1252". VSCode will re-read the file with the correct encoding and display the characters properly. Then save it as UTF-8 to permanently migrate the file to a modern encoding.
Or, to prevent this from happening automatically in the future, add an .editorconfig file to your project:
[*.sql]
charset = windows-1252
The Bigger Lesson
This tiny encoding bug is a perfect case study in how the same input can produce different outputs depending on the assumptions built into your tools — and how those assumptions are often invisible until something breaks.
When you see a ? where a dash should be, the question isn't just "which tool is wrong?" The better question is: what did each tool assume, and why?
The answer usually traces back to a design decision made years ago — a trade-off between portability and platform depth, between being everywhere and being native.
Both VSCode and Visual Studio made reasonable choices. The file just happened to sit exactly on the fault line between them.
Have you run into unexpected behavior differences between VSCode and Visual Studio? I'd love to hear your experience in the comments.