Category Archives: The strange case of…

The strange case of VB6 getting “access denied” building ActiveX DLL running with Registry virtualization

This is one of those posts that I write mainly for myself in the future, but that maybe can be useful for others developers that want or need to run Visual Basic 6.0 without admin rights on Windows 7 or Windows 10.

One of the flavors of my MZ-Tools add-in is for Visual Basic 6.0 (yes, there is still quite a few people using it). Visual Basic 6.0 was created and used heavily in a time (Windows 98-Windows XP) when every Windows user was an administrator, so it was not a problem that when building an ActiveX DLL project some registry entries were added to the HKEY_CLASSES_ROOT (HKCR) registry hive, actually to HKEY_LOCAL_MACHINE (HKLM) internally. On Windows Vista and higher with the User Account Control (UAC) that changed, so if you run Visual Basic 6.0 without admin rights you get this error when building an ActiveX DLL:

A solution is of course to run Visual Basic 6.0 with admin rights, and I have been doing it so for some years. But that requires Visual Studio running also with admin rights, to be able to debug an add-in project loaded on Visual Basic 6.0 (if Visual Studio is not running with admin rights, it prompts you to restart with admin rights, but that’s quite annoying). But running Visual Studio all the time with admin rights only because of Visual Basic 6.0, when all the IDEs (Visual Studio, VBA) of my other add-ins wouldn’t require it, is somewhat overkill, and I really wanted to avoid it.

So, I was aware of the Registry virtualization mechanism of UAC, which Microsoft invented precisely for this scenario: an application that required admin rights to write in some keys of the HKLM\Software key of the Registry but that was not allowed to run with admin rights. Since Registry virtualization was not working for Visual Basic 6.0, my first thought was that it no longer existed on Windows 10 (documents say that apps cannot rely on this mechanism forever since it will be removed in some Windows version), or that it needed to be activated at global level. But no, if you open the Local Security Policy and go to Local Policies > Security Options, the setting exists and file and registry write failures are virtualized by default:

My second thought was that it needed to be activated per-application. But no, all the documents say the opposite, applications can disable registry or file virtualization by adding a .manifest, which signals to the Windows OS that the application is UAC-compliant (and therefore virtualization is not required). And indeed Process Explorer revealed that vb6.exe runs with virtualization:

So, what was happening? Process Monitor to the rescue. I noticed that Visual Basic 6.0 was trying to write to HKCR\TypeLib, getting “access denied”, not directly to HKLM:

But HKCR is actually a merged view of some keys of HKLM and some keys of HKCU, and clearly the failure was due to the HKLM side.

Finally, a closer reading of the page Registry Virtualization on MSDN, section Controlling Registry Virtualization, gave me the clue:  it happens that not all the keys of HKLM\Software are virtualized by default. In particular, the keys and subkeys of HKLM\Software\Classes, which are used when registering an ActiveX component, are not virtualized by default. Fortunately, you can use the reg.exe tool to query and virtualize them.

So I launched an elevated (with admin rights) command prompt and queried the HCKR\Classes\TypeLib (actually HKLM\Software\Classes\TypeLib) in 32-bit Registry (Visual Basic 6.0 is a 32-bit executable) with this command:

reg.exe FLAGS HKLM\Software\Classes\TypeLib QUERY /reg:32

Note: the fact that Visual Basic 6.0 is a 32-bit executable and that Process Monitor showed HKCR\TypeLib (that would belong to 64-bit) and not HKCR\WOW6432Node\TypeLib (that would belong to 32-bit) means that both keys (32-bit and 64-bit) are the same.

I got that it is not virtualized (REG_KEY_DONT_VIRTUALIZED: SET):

To clear that flag I executed the following, that clears the DONT_VIRTUALIZED flag but keeps set the RECURSE_FLAG as before:

reg.exe FLAGS HKLM\Software\Classes\TypeLib SET RECURSE_FLAG /reg:32

Trying to build again the ActiveX DLL causes the same “Error accessing the system registry”, but this time Process Monitor revealed that the problem was in HCKR\WOW6432Node\Interface:

Notice that in this case HCKR\WOW6432Node\Interface is shown, instead of HCKR\Interface, which means that the 32-bit and the 64-bit keys are different.

So I queried the flags with this command:

reg.exe FLAGS HKLM\Software\Classes\Interface QUERY /reg:32

And I got that certainly is not virtualized:

Again, to clear that flag I executed the following, that clears the DONT_VIRTUALIZED flag but keeps set the RECURSE_FLAG as before:

reg.exe FLAGS HKLM\Software\Classes\Interface SET RECURSE_FLAG /reg:32

Trying to build again the ActiveX DLL caused the same “Error accessing the system registry”, but this time Process Monitor revealed that the problem was in HCKR\Interface, so I executed:

reg.exe FLAGS HKLM\Software\Classes\Interface SET RECURSE_FLAG /reg:64

And finally building the ActiveX didn’t cause any error.

VS 2017 RC breaking things: the strange case of MZ-Tools 8.0 not changing user interface language

I received four days ago a problem report from a German user of my MZ-Tools add-in for VBA, explaining that after updating to the last build 8.0.0.120, the language of the user interface was always English, despite MZ-Tools being configured to use the language of the Regional Settings, and being these German. Nobody else had reported this and I was unable to reproduce it.

At first I thought that there was a problem in the logic detecting the language of the Regional Settings, but after some verification that was not the case, the problem happened also selecting German directly (and the logic to detect the language of the Regional Settings hasn’t changed for years).

Then I requested him to send me his files and settings. I copied them to my computer but I was unable to reproduce the problem, German was used as language.

Then I sent him all the builds between the last working build and the last (failing) build, so we determined which exact build introduced the problem.

Using source control I could determine that the first failing build used a somewhat different approach to generate the resource dlls used by MZ-Tools, because I am these weeks migrating from a custom builder for MZ-Tools to 100% MSBuild. But the new approach seemed very fine to me, so it should be something else, and I had a suspicion.

In the past years MZ-Tools used an embedded xml file with all languages, and while it worked fine, in recent times I have a tendency to adopt standard Microsoft approaches, so I switched to .resx/resource dlls (despite my dislike for file proliferation). Soon I became familiar with an issue with .resource.dll files: if for some reason the Common Language Runtime (CLR) of the .NET Framework fails to load them, the failure is silent and the user interface defaults to English. This happened in the case that I described in the post The strange case of localized resources not being loaded, where I was failing to re-sign the resource dlls after obfuscating and re-signing the main assembly. Since then I have learned more about obfuscation and signing and I no longer use delay-signing.

Notice that when resource dlls fail to load, three errors are happening:

  • Something is generating bad resource dlls (that can’t be loaded by the CLR).
  • PEVerify.exe is not being used to verify the resource dlls during the MSBuild script, or it is not reporting the problem.
  • The error is not detected at runtime (it’s a silent error), although it can be detected with the Assembly Binding Log Viewer (fuslogvw.exe).

Rather than focusing on the first item to solve the problem, I like to focus first on providing early detection of problems. In my case I noticed that my MSBuild script was not using PEVerify.exe to verify the resource dlls (it was used only against main assemblies), so I added that verification to the build script. Strangely, the verification was successful. But I noticed that I was using the $(TargetFrameworkSDKToolsDirectory) MSBuild variable, that uses .NET Framework 4.0. MZ-Tools 8.0 for VBA is built against .NET Framework 2.0, so I changed the MSBuild script to use the peverify.exe tool of .NET Framework 2.0. And then, finally, I got this:

“The assembly is built by a runtime newer than the currently loaded runtime, and cannot be loaded.”

What? I used the disassembler ildasm.exe, and surprise, surprise, the MZTools8PlugIn.resources.dll was referencing mscorlib version 4.0, despite MZTools8PlugIn.dll being targeting .NET Framework 2.0:

mscorlib4

I reproduced the problem creating a Class Library project and adding a resource file. What could be causing this? Quickly I thought about other thing that changed on my computer in the last weeks: I installed Visual Studio 2017 RC to add support for that IDE in MZ-Tools 8.0 for Visual Studio. Latest builds of MZ-Tools are compiled with Visual Studio 2017 RC, but I noticed that the problem also happened using Visual Studio 2015 on the same computer. I was quite sure that this was not the case in the past, as I verified on a second computer with Visual Studio 2015 without Visual Studio 2017 RC. So, this is a problem introduced by Visual Studio 2017 RC, and I have reported it here:

When VS 2017 RC is installed, resources.dll is generated from .resx file using mscorlib 4.0 even when project targets .NET 2.0

I don’t have a workaround yet (other than using a computer without Visual Studio 2017 RC), but at least now I know the culprit and I have a MSBuild script that will detect all future cases at build time.

The last piece to be explained is why I was unable to reproduce the issue on my computer. The reason is that the user is using Windows 7, where MZ-Tools 8.0 for VBA runs on the installed .NET Framework 2.0 (and therefore cannot load .NET 4.0 resource dlls), while I am using Windows 10, where MZ-Tools 8.0 for VBA runs on the installed .NET Framework 4.0, which is happy to load .NET 4.0 resource dlls.

The strange case of VBA / VB6 editor prompting to repair Visual Studio (revisited 3rd time)

This is the nth time that I find this problem (and I am not the only one), so I am going to document in this 3rd post another cause for it, that happens on Window 10. The problem is that suddenly, the VBA editor of Office or VB6 prompts you to repair some version of Visual Studio. In this case, VB6 prompts to repair Visual Studio 2005 (that I have installed on my main development machine) but it can also happen with Visual Studio 2010, etc.:

vb6prompttorepair

I already talked about this back in 2012, where the root cause was the installation of Visual Studio from a removable hard disk (since then I always copy the setups of Visual Studio to the fixed hard drive before installing) and in 2014, where the cause was the deletion of partition D: of the hard disk (actually a variation of the previous cause). In both cases I was unable to fix the problem until I uninstalled all versions of Visual Studio and reinstalled them, which for a developer of extensions like me that need all Visual Studio versions to test is the most painful experience.

Since I adopted Windows 10 as my main OS, I have seen this problem twice, in 2015 and in 2016, and it has the same cause in both cases as I will explain, but different to the ones of 2014 and 2012. To diagnose this issue, you need to follow the procedure explained by Heath Stewart in his blog. So I started looking for events with ID 1001:

eventviewer1001

As you can see, the error doesn’t tell you exactly which is missing. The events with ID 1004 provide the actual cause:

eventviewer1004

So, the file C:\Windows\Microsoft.NET\Framework\v2.0.50727\RedistList\VSList.xml is missing. This is a file likely installed by Visual Studio inside a folder that belongs to the .NET Framework, not to Visual Studio. I would say this is a bad idea. And it is not the only .xml file installed by Visual Studio, there are a lot of them.

But which is the root cause that makes that a file suddenly disappears from that folder? It is the installation of the major updates of Windows 10: Update “1511” in November 2015 and “Anniversary Update” 1607 in October 2016. As you know, the next one announced a couple of days ago will be “Creators Update” in 2017 and the issue will reappear in 3D ;-). Each time that a major Windows 10 update is installed, the previous Windows installation is stored in a backup folder “Windows.old” (which includes the .NET Frameworks folders, and the .xml files in question). It seems that the Windows 10 update is smart to detect that you had .NET Framework 2.0 installed (it’s an optional feature of Windows 10, not installed by default) and it installs it for you, but not smart enough to detect the missing files. So, alas, the .xml files are not restored and cause this issue. In 2015 I still had the Windows.old backup folder and I was able to restore all the missing .xml files using BeyondCompare, the folder comparison utility that I love. Now in 2016 I deleted the Windows.old backup folder before I realized the issue, so I had to provide the .msi files to the prompt dialog, copying them to the hard disk before, of course!

I hope this helps you if you encounter this issue.

The strange case of System.NotImplementedException “The method or operation is not implemented.” using SQL Server database projects

Yesterday I got a bug report from a user of my MZ-Tools package for Visual Studio 2015:

System.NotImplementedException “The method or operation is not implemented.”

The exception happened calling the EnvDTE.ProjectItem.IsOpen Property (viewKind) method when using SQL Server Database Projects provided by SQL Server Data Tools (SSDT):

databaseproject

As you may know, SQL Server Database projects are quite good automation-friendly projects, that is, they implement EnvDTE interfaces (EnvDTE.Project, EnvDTE.ProjectItem, etc.) so you don’t need to use the low level IVsHierarchy, etc. interfaces of Visual Studio. But somehow, they missed to implement something somewhere.

I tried to reproduce the problem on my side to no avail. My extension worked happily with the .sql files (EnvDTE.ProjectItem) of that kind of project. So, which kind of EnvDTE.ProjectItem didn’t implement the IsOpen property? The user sent me a helpful clue: he was using a database reference. I was not aware of that thing, but certainly in SQL Server database projects you can add references to databases:

databasereference

And as soon as I added a reference to the “master” database, I could reproduce the problem. So, my extension was treating the ProjectItem nodes below the References node as files that could be “opened”, and the ProjectItem.IsOpen method was not implemented (which is certainly right). But, how is that the nodes below the References node were being processed for database projects (which is incorrect) while they were not processed for regular (C#, VB.NET) projects?. The reason is that for regular projects, the EnvDTE.Project.ProjectItems collection of the project doesn’t return the References node, which is correct. For database projects, the EnvDTE.Project.ProjectItems collection does return the References node, which is a bug. I wanted to verify that the internal implementation of database projects has this bug so I used .NET Reflector to see the internals of the Microsoft.VisualStudio.Data.Tools.Package.dll package and I found that there is a ReferenceContainerNode type that has a GetAutomationObject that returns an OAReferenceFolderItem instance:

getautomationobject

There shouldn’t be a folder for references in the automation model (EnvDTE). The automation model provides the VSLangProj.References collection to handle the references of a project. I won’t report the bug because there is an almost zero chance of Microsoft fixing it, and anyway I needed to find a fix for the current version of the SQL Server Data Tools, which took me to the next problem: how can an extension identify reliably the “References” ProjectItem?.

I could check if the Name property of the ProjectItem is “References”, but that approach is always prone to errors due to localization (in Spanish would be “Referencias”). The SQL Server Data Tools doesn’t seem to be localized, but I wanted to investigate another approaches. It happens that the IVsHierarchy of the References node implements an interface named IReferenceContainer. So, one could cast that node to IReferenceContainer and if the cast is successful then it’s the References node. Alas, that interface is declared internally in database projects, and in MPF.Project.NonShipping.dll for other projects. At the end, I opted for getting the EnvDTE.ProjectItem.Object property, convert it to a string, and compare it to “Microsoft.VisualStudio.Data.Tools.Package.Internal.OAReferences”, which is as bad as comparing the Name to “References”, but anyway, it works for now.

Another two strange cases of MZ-Tools 8.0 for VBA could not be loaded

It seems that the path to load a .NET-based add-in in the VBA editor of Office is full of mines. In the last couple of days I have discovered two new scenarios (apart from this and this) that can cause “<Add-In> could not be loaded”. I will document them here for future reference for me or other people:

This first one happens if you are not using a COM Shim, you are using .NET Framework 2.0 and the version of Office is 2000, 2002 (XP) or 2003. It is due to a lockback policy that “prevents the .NET Framework 2.0 common language runtime (CLR) from initializing when the .NET Framework 2.0 is hosted in either the Word process space or the Excel process space. The policy restriction limits Word and Excel from loading versions of the .NET Framework that are later than version 1.1. Therefore, the .NET Framework 2.0 assemblies cannot load”. It is documented in the Knowledge Base article You cannot load a .NET Framework 2.0 assembly from Visual Basic for Applications in Word 2003 and earlier versions, or in Excel 2003 and earlier versions and fortunately has a solution through a registry key HKEY_CLASSES_ROOT\Interface\ {000C0601-0000-0000-C000-000000000046} that can be created. I was aware of this problem since long time ago and my setup created that registry key if Office 2003 or 2002 was detected. However, I noticed that my add-in was failing for Office 2000 (yes, I have users running Office 2000 on old PCs) because I added support for Office 2000 later and I didn’t update the setup to check that version. This case is difficult to diagnose because the following script works perfectly (I mean, the problem only manifests loading the assembly in Office):

Dim o
Set o = CreateObject("MZTools8VBA")

The other problem was also hard to diagnose. It happened when using a COM Shim to load the CLR using code from the COM Shim wizards provided by Microsoft, which calls the CorBindToRuntimeEx function passing NULL to the first pwszVersion parameter. A NULL value means to load the highest CLR installed below CLR 4.0, which should be CLR 2.0. However, on Windows XP I got HRESULT 0x80131700. Fortunately I found on a comments of the post COM Shim CLR Loader Bug that the problem could be solved passing the specific version “v2.0.50727”, which certainly fixed the problem for me.

The strange case of System.InvalidCastException (“Unable to cast COM object of type ‘System.__ComObject’ to class type System.Windows.Forms.UserControl”) showing toolwindow

In my last post I explained a problem that can happen when a .NET-based add-in for the VBA/VB6 editor that uses a COM Shim tries to use any Common Language Runtime (CLR) that it’s installed on the target machine (CLR 2.0 or CLR 4.0), assuming that the add-in is built with .NET Framework 2.0/3.0/3.5 and therefore can use CLR 2.0 or CLR 4.0. This post is about other problem that can happen.

The ideal approach, that was the first that I implemented, is that the COM Shim uses at run-time the first detected CLR that it supports. In my case the logic was:

  • First it detects if CLR 2.0 is installed. If so, use it.
  • If not, it detects if CLR 4.0 is installed. If so, uses it.

This would be the best approach because at any given time, it uses the available(s) CLRs. The default CLRs installed by Windows are:

  • Windows XP: Neither CLR 2.0 nor CLR 4.0
  • Windows Vista, Windows 7: CLR 2.0
  • Windows 8, Windows 8.1, Windows 10: CLR 4.0

And life would be great if that were the whole picture. However, most add-ins need to create and show toolwindows. For a .NET-based add-in, this is somewhat tricky because toolwindows need ActiveX UserDocuments, and .NET doesn’t support UserDocuments. However, a UserControl can be used instead with some tricks as I explained in HOWTO: Create a toolwindow for the VBA editor of Office from an add-in with Visual Studio .NET. The UserControl, being .NET-based, needs to be registered as a COM component, and .NET-based COM components need to specify the required runtime version of the .NET Framework in the Windows registry, in the RuntimeVersion value inside the InProcServer32 key:

HKEY_CLASSES_ROOT\progid
   (default) = progId
HKEY_CLASSES_ROOT\progid\CLSID
   (default) = clsid
HKEY_CLASSES_ROOT\CLSID\{clsid}
   (default) = progid
HKEY_CLASSES_ROOT\CLSID\{clsid}\InProcServer32
   (default) = mscoree.dll
   Class = ClassName
   ThreadingModel = Both
   Assembly = Stringized_assembly_reference
   Codebase = path_of_private_assembly
   RuntimeVersion = version_of_the_runtime
...

toolwindowcontrolregistration

This registration of the usercontrol used by the toolwindow must be done by the installer at setup-time. And I used in the installer the same logic than in the COM Shim: first it detects CLR 2.0 and, if not available, then it detects CLR 4.0.

However, a couple of my users reported this exception when clicking a button that should show a toolwindow:

System.InvalidCastException: Unable to cast COM object of type ‘System.__ComObject’ to class type ‘System.Windows.Forms.UserControl’. Instances of types that represent COM components cannot be cast to types that do not represent COM components; however they can be cast to interfaces as long as the underlying COM component supports QueryInterface calls for the IID of the interface.

Toolwindows are created calling the CreateToolWindow function, that receives in the second parameter the ProgId of the ActiveX UserDocument that the function will create internally and return as output last parameter:

Windows.CreateToolWindow (AddInInst, ProgID, Caption, GuidPosition, DocObj) As Window

And that exception happened in a line of code that casts the usercontrol instance returned by the last output parameter (which should be a UserControl instance) to the System.Windows.Forms.UserControl type. Why?

It took me a while to discover the cause (the scenarios are so messy that often I need to go walking or swimming to get clarity thinking about the problem…). But at some point I realized that there were some scenarios where the runtime version of the usercontrol and the runtime version of the COM Shim would mismatch:

The first one is that you have Windows 10, which by default provides .NET Framework 4.0 (CLR 4.0) and not .NET Framework 2.0/3.0/3.5 (CLR 2.0). The user runs the setup of the add-in. The usercontrol is registered with the only runtime available (v4.0.30319). When the add-in is loaded the COM Shim uses the only runtime available, also v4.0.30319. So far so good. However, one day the user installs .NET Framework 2.0/3.0/3.5, which is an optional feature of Windows 10:

windowsoptionalfeatures

In this case, the usercontrol remains registered with runtime version v4.0.30319 (unless the user runs the installer again) but the COM Shim, that detects CLRs at runtime, not at setup time, would use CLR 2.0! (because its logic gives it preference over v4.0.30319). And this causes the System.InvalidCastException, because the CreateToolwindow returns an instance of a CLR 4.0 usercontrol. Since such thing cannot enter the CLR 2.0 of the COM Shim, a System.__ComObject wrapper is created, that cannot be cast to a UserControl type.

The second scenario is that you have Windows 10, which by default provides .NET Framework 4.0 (CLR 4.0) but the user has installed .NET Framework 2.0/3.0/3.5 (CLR 2.0). The user runs the setup of the add-in. The usercontrol is registered with the runtime v2.0.50727 (because the logic gives it preference over v4.0.30319). When the add-in is loaded the COM Shim uses also the preferred runtime v2.0.50727 over v4.0.30319. So far so good. However, one day the user uninstalls the optional .NET Framework 2.0/3.0/3.5. In this case, the usercontrol remains registered with runtime version v2.0.50727 (unless the user runs the installer again) but the COM Shim, that detects CLRs at runtime, not at setup time, would use CLR 4.0 instead of CLR 2.0! This causes a different exception, though.

So I thought a different approach: rather than using whatever CLR is installed on the machine (with preference for CLR 2.0 over CLR 4.0), I use the default CLR that should be installed on each Windows version:

  • Windows XP: none. But the setup enforces that .NET Framework 2.0 must be installed.
  • Windows Vista, Windows 7: .Net Framework 2.0
  • Windows 8, Windows 8.1, Windows 10: .Net Framework 4.0

This approach makes the COM Shim immune to installation/uninstallation of additional .NET Framework installations (because default .NET Frameworks are OS components and cannot be uninstalled, only optional .NET Frameworks can be uninstalled).

And while this approach seemed fine, another two users reported the same System.InvalidCastException, both cases on Windows 10 using VB6.  More walking and swimming :-). The reason is that VB6 can be set to run in compatibility mode with Windows XP:

windowsxpcompatibility

What happens if a user is running Windows 7 and sets VB6 with compatibility mode with Windows XP? The setup was run on Windows 7, so the usercontrol is registered with runtime version v2.0.50727 (the default for Windows 7). When VB6 in compatibility mode loads the COM Shim, it thinks that it is running on Windows XP and uses the .NET Framework 2.0 that the setup required. So nothing bad happens.

But what happens if the user is running Windows 10 and sets VB6 with compatibility mode with Windows XP? The setup was run on Windows 10 (not in compatibility mode), so the usercontrol is registered with runtime version v4.0.30319 (the default for Windows 10). When VB6 in compatibility mode loads the COM Shim, it thinks that it is running on Windows XP and uses the .NET Framework 2.0 that the setup required. So a System.InvalidCastException happens when performing the cast operation due to the mix of runtime versions.

The best solution, that I hope that fixes this problem for good, is to make the COM Shim to read the registered runtime version of the usercontrol of the toolwindow from the Windows registry, and use the matching CLR. With this approach, it doesn’t matter what Windows version the COM Shim thinks it is running on, because it won’t use the default CLR of that Windows version, but the one that was determined at setup-time.

Update (September 12, 2016): it happens that there is a way to register a .NET component as a COM component indicating that it can work with CLR 2.0 and CLR 4.0. Instead of using the RuntimeVersion name mentioned above, you can use the SupportedRuntimeVersions name and provide a list of CLRs in order of preference:

HKEY_CLASSES_ROOT\progid
   (default) = progId
HKEY_CLASSES_ROOT\progid\CLSID
   (default) = clsid
HKEY_CLASSES_ROOT\CLSID\{clsid}
   (default) = progid
HKEY_CLASSES_ROOT\CLSID\{clsid}\InProcServer32
   (default) = mscoree.dll
   Class = ClassName
   ThreadingModel = Both
   Assembly = Stringized_assembly_reference
   Codebase = path_of_private_assembly
   SupportedRuntimeVersions = v2.0.50727;v4.0
...

See the post In-Proc SxS and Migration Quick Start. So, original approach for the COM Shim would work:

  • First it detects if CLR 2.0 is installed. If so, use it.
  • If not, it detects if CLR 4.0 is installed. If so, uses it.

The strange case of mscoree.dll and the CLRCreateInstance function

Since I started three months ago to use a COM Shim for my .NET-based MZ-Tools 8.0 for VBA/VB6 to achieve isolation and .NET CLR independence, nobody had reported a single problem. However, last week two different problems were reported. This post is about the first one. The error was that on Windows XP, MZ-Tools 8.0 for VB/VBA failed to load. This dreaded error message:

MZTools8CouldNotBeLoaded

Fortunately, the user reported that installing .NET Framework 4.0 the problem was solved, but why?

The error message is hiding the actual error code and description (Microsoft started to show that crucial piece of information when an add-in fails to load in Visual Studio .NET 2002). I was able to reproduce the problem so I didn’t need to contact the user for further information. An approach to know why an add-in is failing to load on any IDE is to use Process Monitor to capture disk and registry activity and then guess what is failing (some missing registry entry or file, etc.). Unfortunately the current version of Process Monitor doesn’t support Windows XP, and although I couldn’t find an earlier version to download, I found the Registry Monitor and File Monitor utilities (that later became unified in Process Monitor) and they worked on Windows XP.  However, I couldn’t guess the problem. I always forget that the easiest way to know the actual error is to create a VBScript with this code that creates a COM object with the ProgId of the add-in (typically “MyAddIn.Connect”):

Dim o
Set o = CreateObject("MZTools8VBA")

And executing that script shows the actual problem:

ProcedureEntryNotFound

ProcedureEntryNotFoundScript

So, the problem was that the CLRCreateInstance function (used by the COM Shim to load the Common Language Runtime, CLR) was not found in mscoree.dll. At this point I knew I needed another tool: depends.exe (Dependency Walker).

The CLRCreateInstance function was introduced in .NET Framework 4.0 to load the CLR 4.0. The CLR 2.0 used by .NET Frameworks 2.0, 3.0 and 3.5 is loaded with a different function. My COM Shim has this logic:

  • First it detects if .NET Framework 2.0 is installed. If so, it uses the old API to load CLR 2.0.
  • If not, it detects if .NET Framework 4.0 is installed. If so, it uses the new CLRCreateInstance API to load CLR 4.0.

Note: this logic that uses run-time detection of the CLR rather than setup-time detection is flawed by a different reason that caused the second reported problem (for another post).

Since my setup requires .NET Framework 2.0 on Windows XP, I knew that the CLRCreateInstance function was not called on Windows XP. However, since my COM Shim dll linked it at compile-time:

COMShimOld

the whole dll of the COM Shim failed to load if that function is missing in mscoree.dll (it seems that my knowledge of C/C++ linking is very limited, I thought I would get a runtime error only if the COM Shim tried to call that missing function…).

This explained why installing .NET Framework 4.0 solved the problem: even if the COM Shim still used CLR 2.0 (by the logic above), the mscoree.dll updated by .NET Framework 4.0 provided the missing CLRCreateInstance function and the COM Shim dll didn’t fail to load.

But a question remained that got me puzzled for several days: nobody had reported this problem on Windows 7, that lacks .NET Framework 4.0 (it installs only .NET Framework 3.5 / CLR 2.0), in these three months. And since my setup doesn’t require .NET Framework 4.0 on Windows 7, I was sure that not all users have .NET Framework 4.0 installed. I should have received tons of bug reports with the same problem. But not a single one… The first thing I did was to check the version of mscoree.dll of my virtual machines with Windows 7. First surprise: they showed 4.0.xxxx, although .NET Framework 4.0 was not installed:

mscoreeWindows7WithSP1

Then I used depends.exe to check that the mscoree.dll of Windows 7 certainly exports the CLRCreateInstance function:

mscoreeExportedFunctionsWindows7WithSP1

So, how is that Windows 7, lacking .NET Framework 4.0, has a mscoree.dll version 4.0.xxxx that provides the CLRCreateInstance method to load the CLR 4.0?

At this point I decided to start from scratch: I created a virtual machine with Windows 7 RTM, and suprise, surprise, mscoree.dll was version 2.0.xxxx:

mscoreeWindows7WithoutSP1

and certainly lacked the CLRCreateInstance method:

mscoreeExportedFunctionsWindows7WithoutSP1

And of course, installing VB6 or Office and MZ-Tools 8.0 reproduced the problem that happened on Windows XP.

It was only after I applied SP1 to Windows 7, that mscoree.dll was updated to version 4.0.xxxx. So, it seems that Microsoft decided to include the mscoree.dll version 4.0.xxxx of .NET Framework 4.0 in the service pack 1 of Windows 7, even if that service pack doesn’t include .NET Framework 4.0.

Of course the best solution is to use dynamic linking at run-time using GetProcAddress so that the COM Shim can be loaded even if the CLRCreateInstance function is missing:

COMShimNew

For the record, Windows 10 uses version 10.0.xxxx of mscoree.dll:

mscoreeWindows10

Case closed.

The strange case of MZ-Tools 8.0 crashing the VBA editor of Autodesk Inventor

Recently I got a bug report from a customer of my MZ-Tools 8.0 add-in for VBA stating that it crashed the VBA editor of Autodesk Inventor 2016. If he uninstalled MZ-Tools, the problem disappeared. This user has been using MZ-Tools 8.0 for months but now, suddenly, this problem appeared.

I downloaded the trial version of Autodesk Inventor 2017 and I was able to reproduce the problem. This was the first window of the crash:

AutodeskCrash

Clicking the View Report Details more info was shown:

AutodeskCrashDetails

So, it was an unhandled exception. The dmpuserinfo.xml contained the stack trace:

AutodeskCrashStackTrace

So, when MZ-Tools shows a modal window, Autodesk Inventor tries to set a “quick access toolbar gradient color” using a function that crashes the app. Initially I thought (and so it was reported) that the crash happened only when MZ-Tools was set to load on startup, but not when loaded manually using the Add-In Manager. However, I realized that it happened at any time that MZ-Tools tried to show a modal dialog.

What could be causing this after months of normal usage? My main suspect was the COM Shim that I introduced in build 8.0.0.77 (June 1) to provide proper isolation from other .NET-based add-ins for the VBA editor. And certainly the hypothesis was correct. When I ran MZ-Tools without the COM Shim, the problem didn’t happen. So, it seems that WinForms dialogs don’t play well when loaded from a different AppDomain. The several Office apps and versions don’t suffer this problem, but Autodesk Inventor does. Although I reported the problem to Autodesk, I guess that likely they won’t fix it, or anyway the existing released Autodesk Inventor 2016 and 2017 builds will persist with the problem. So, as a workaround, the new setup of MZ-Tools 8.0 for VBA will detect if Autodesk products are installed. In that case, the setup won’t register MZ-Tools using the COM Shim.

The strange case of MZ-Tools 8.0 for VBA could not be loaded

Yesterday I got this error when loading my add-in MZ-Tools 8.0 for VBA:

MZTools8VBACouldNotBeLoaded

I hate that error. The VBA and VB6 IDEs hide valuable information about why an add-in couldn’t be loaded. In Visual Studio you get at least a COM error number (the error message is almost always <Unknown error> for COM errors):

MZTools8VSCouldNotBeLoaded

And you have this article of mine explaining most causes given an error number: HOWTO: Troubleshooting Visual Studio and Office add-ins.

But for VB6/VBA, you are clueless. I have got that error many times because my MZ-Tools add-in is supported in several IDEs using different approaches for registration for the end user (using a setup) and for myself during development/debugging (using PowerShell scripts for registration/unregistration).

For the end user (using a setup):

  • MZ-Tools 8.0 for VBA uses per-user COM-registration, not requiring admin rights to install, which is important for many Office users in corporate environments. Per-user COM registration works because Office doesn’t run with admin rights by default (an executable running elevated can not see per-user COM-registration, for security reasons).
  • MZ-Tools 8.0 for VB6/VB5 uses machine-wide COM-registration, requiring admin rights to install, because per-user COM-registration wouldn’t work since VB6 runs elevated by default (to be able to register ActiveX Dlls when compiling).

For development/debugging (using PowerShell scripts):

  • MZ-Tools 8.0 for VB6/VB5 uses machine-wide COM-registration, because of the reason explained above. That means that I must run Visual Studio always elevated, so it can launch vb6.exe for debugging without annoying prompts for elevation of Visual Studio with a restart.
  • MZ-Tools 8.0 for VBA uses also machine-wide COM-registration, because since I run Visual Studio always elevated, Office runs always elevated during debugging, and therefore per-user COM-registration wouldn’t work (remember, a process running elevated cannot see per-user COM-registration).

All this means that any error using the wrong registration or running Visual Studio, VB6 or Office with the wrong elevation (or lack of), and I get the dreaded “MZ-Tools 8.0 could not be loaded”.

In those cases, I need to use Process Monitor (see HOWTO: Using the Process Monitor (ProcMon) tool to diagnose Visual Studio add-ins problems) to try to guess which registry entry or file is causing a problem. In this case, I got this:

PathNotFound

So, the dll of the add-in was not found. But, wait a minute: it is looking for it in the path “C:\Users\<user>\AppData\Local\MZTools Software\MZTools8VBA\”, which is used for the end user (setup), not in the path used during development. I went to the Control Panel, Program and Features, and MZ-Tools 8.0 was not installed!. So, somehow the last uninstallation of the setup failed silently, removing the file but leaving the COM-registration in the Windows registry. How could that be? I test the setups and PowerShell scripts exhaustively when created/modified and they were working correctly for weeks since the last changes.

Then, suddenly, I remembered something. I am using Windows 10, and I always use the “Control Panel”, “Program and Features” item to uninstall the software, because I like colors and I am not a fan of the new bland “Settings” > “System” > “Apps and features” user interface of Windows 10. But the day before, I used the “Apps and features” to uninstall MZ-Tools 8.0 for VBA, maybe because I guess that the Control Panel will be removed at some point since it seems a remaining duplicated thing. So, after some manual cleaning of registry entries, I tried again to install and uninstall using the setup (using “Settings” > “System” > “Apps and features”) and then to register for development using the PowerShell script and the problem was reproduced. One try again and I realized the cause:

  • If you have a setup that doesn’t require admin rights to install, the  “Control Panel”, “Program and Features” doesn’t require admin rights to uninstall either, which is correct.
  • However, the “Settings” > “System” > “Apps and features” prompts for admin rights to uninstall even if the setup was installed without admin rights, which seems incorrect to me.

It happens that some users of MZ-Tools 8.0 for VBA run Office with admin rights (I don’t know really why they want that), and therefore if MZ-Tools 8.0 for VBA runs with per-user COM-registration, it doesn’t work. So I changed the setup of MZ-Tools 8.0 for VBA some months ago to behave as follows:

  • By default, the setup runs without admin rights. In that case, the setup registers the add-in using per-user COM-registration.
  • If you run Office with admin rights, you can force the setup to use machine-wide COM-registration (so it works) running it with admin rights (using the “Run as administrator” context menu). The setup detects that it is running with admin rights and uses machine-wide COM-registration instead of the default per-user COM-registration. Conversely, the uninstaller detects if it is running with admin rights and in that case removes machine-wide COM-registration; otherwise, it removes per-user COM-registration.

And then it happens that on Windows 10, if you use the “Settings” > “System” > “Apps and features” to uninstall, the uninstaller is elevated to use admin rights (even if it was installed without admin rights) and it removes the non-existent machine-wide COM-registration, leaving intact the existing per-user COM-registration that was created when installed without admin rights. When the PowerShell script registers the add-in using machine-wide COM-registration, two COM-registrations are in place. If I run Office with admin rights, the machine-wide COM-registration is used and it works. But if I run Office without admin rights, the per-user COM-registration that shouldn’t be there takes precedence, and points to a file that was uninstalled, so the error.

Now I have to modify the uninstaller to remove also the per-user COM-registration even if running with admin rights.

The strange case of MZ-Tools keeping disabled

I have received a couple of issues in the last days from users of my MZ-Tools package, reporting that it keeps disabled, they have to enable it, and when restarting Visual Studio 2015 it happens again.

I haven’t seen that issue, so I could not investigate on my computer, but searching the web I found this post of MVP fellow David Gardiner that explains that the issue can happen to any extension and he provides a solution:

Web Essentials 2015 extension keeps being disabled

So far my users have confirmed the solution. Thanks David!