I started programming in the old times of 16-bit MS-DOS, where you had only 640 KB of memory, and you had concepts like “expanded memory” and “extended memory” and memory bank switching to use more than 640 KB and then wonderful things like the Watcom C++ compiler that allowed you to create linear 32-bit MS-DOS programs to use all the available memory, not just 640 KB or 1 GB MB (oops!). When Windows (95) became a 32-bit OS and I learned at college about 32 bit pointers, it seemed that 4 GB of memory will be enough forever. Guess what? Some weeks ago I got “out of memory” errors from Visual Studio when trying some features of my MZ-Tools add-in that operate on every file of a solution. Specifically, the problem happened when iterating the code elements of the EnvDTE.ProjectItem.FileCodeModel of every file of the solution. My computer had 4 GB of RAM so I was surprised by this error (later I learned that the amount of RAM is irrelevant for most “out of memory” errors in modern virtual memory operating systems).
The first reference that I found on the web about this error was in the MSDN VSX forum:
ProjectItem.FileCodeModel causes memory fragmentation?
http://social.microsoft.com/Forums/en-US/vsx/thread/9d6c1d17-63c8-40c4-b6bc-e7f0ead65c3d
Notice that while MSFT people explained that this is due to COM objects in memory wrapped by .NET objects that are not released, the explanation does not solve the problem, neither the manipulation of the Garbage Collector (GC). I didn’t try using EnvDTE.Project.CodeModel because it didn’t meet my needs of “file by file” processing.
Then I found that mine was not the only extension for Visual Studio experiencing this problem. Resharper was also experiencing it and provided some tools to fix it. But it was non until I read this post when I learned about the cause and (kind of) solution for this problem:
Hacking Visual Studio to Use More Than 2Gigabytes of Memory
http://old.stevenharman.net/blog/archive/2008/04/29/hacking-visual-studio-to-use-more-than-2gigabytes-of-memory.aspx
So, the problem is not the lack of physical memory (RAM) or virtual memory, but the address space of 32-bit processes, which is limited to 4 GB and half of it is reserved for the OS (kernel mode) so the process has only 2 GB for it (user mode). And using the file code model of every file of a solution can cause Visual Studio to reach the limit of 2 GB of its user-mode address space. Using the /3GB option in the boot.ini file allows 3 GB of address space for the user mode part of the process and 1 GB for the kernel mode part, which can cause other problems because 1 GB is little these days for some video cards, etc.
Also, to take advantage of the 3 GB address space, a process must be “marked” to indicate the OS that it can use all those memory addresses. This is done with the LARGEADDRESSAWARE flag in the executable image that Visual Studio 2008 does have set, but Visual Studio 2005 doesn’t (although you can patch it with the editbin utility as explained in the link above).
In my case, using the /3GB option in boot.ini with VS 2008 “solved” the problem and using the Task Manager I saw devenv.exe passing the limit of 2 GB without causing the “OutOfMemory” exception. I say “solved” because this is not really a happy solution, it is not something that you can patch in your add-in, you need to show a message explaining the user to either use Windows 64-bit (which gives 32-bit processes the full 4 GB address space for user-mode) or to use the /3GB option with Windows 32-bit (that can cause other problems, in my case with two other applications).
So, 32-bit pointers and 4 GB are not enough these days… maybe with 64-bit pointers we’ll have these problems solved forever.
There is a bunch of very interesting resources on the web about all this, starting by the Raymond Chen’s series:
Summary of the recent spate of /3GB articles
http://blogs.msdn.com/oldnewthing/archive/2004/08/22/218527.aspx
and this one of today:
“Out Of Memory” Does Not Refer to Physical Memory
http://blogs.msdn.com/ericlippert/archive/2009/06/08/out-of-memory-does-not-refer-to-physical-memory.aspx