Workaround for VS 2010 Beta 2 Bug: CommandBarButton.Style not honored, always Icon + Caption

Back in VS.NET 2002/2003 the only way to change the style (Icon only, or Icon + Caption) of a CommandBarButton created from a Command was to cast the CommandBarControl returned by Command.AddControl to CommandBarButton and use the CommandBarButton.Style property.

If you use that approach in VS 2010 Beta 2 you will find this bug that I already mentioned:

VS 2010 Beta 2 Bug: CommandBarButton.Style not honored, always Icon + Caption
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=500403

That means that if the toolbar of your add-in has 30 buttons which should be shown as “only icon”, due to this bug they will shown as “icon + caption” and you hardly will see 7 of them…

Fortunately there is a workaround thanks to the EnvDTE80.Commands2.AddNamedCommand2 method that was introduced in VS 2005. So you can:

  • Cast the EnvDTE.DTE.Commands (or EnvDTE80.DTE2.Commands) property to EnvDTE80.Commands2
  • Call the Commands2.AddNamedCommand2 method passing in the CommandStyleFlags parameter the EnvDTE80.vsCommandStyle.vsCommandStylePict value

I hope this helps.

Yet another critical VS 2010 Beta 2 bug in CommandBars: this one causes VS to hang, nothing less

This one happens when an add-in:

– Creates a temporary toolbar using CommandBars.Add

– Creates a temporary CommandBarPopup on that toolbar

– Creates a command and tries to add a CommandBarButton from that command to that CommandBarPopup using Command.Add(CommandBarPopup.CommandBar)

The bug report is here:

VS 2010 Beta 2 Bug: adding CommandBarButton from Command to CommandbarPopup on Toolbar causes VS to hang

https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=503871

The (VB.NET) code to reproduce the problem is:

Imports System
Imports Microsoft.VisualStudio.CommandBars
Imports Extensibility
Imports EnvDTE
Imports EnvDTE80

Public Class Connect
   Implements IDTExtensibility2

   Private _applicationObject As DTE2
   Private _addInInstance As AddIn
   Private _myCommandBarToolbar As CommandBar
   Private _myCommandBarPopup As CommandBarPopup

   Public Sub OnConnection(ByVal application As Object, ByVal connectMode As ext_ConnectMode, ByVal addInInst As Object, ByRef custom As Array) Implements IDTExtensibility2.OnConnection

      _applicationObject = CType(application, DTE2)
      _addInInstance = CType(addInInst, AddIn)

      Select Case connectMode

         Case ext_ConnectMode.ext_cm_AfterStartup
            InitializeAddIn()

         Case ext_ConnectMode.ext_cm_Startup
            ' OnStartupComplete will be called

      End Select

   End Sub

   Public Sub OnStartupComplete(ByRef custom As Array) Implements IDTExtensibility2.OnStartupComplete
      InitializeAddIn()
   End Sub

   Private Sub InitializeAddIn()

      Const MY_COMMAND_NAME As String = "MyCommand"

      Dim command As Command = Nothing
      Dim commandBars As CommandBars
      Dim commandBarControlObject As Object

      Try

         commandBars = CType(_applicationObject.CommandBars(), CommandBars)

         _myCommandBarToolbar = commandBars.Add(Name:="My toolbar", Position:=MsoBarPosition.msoBarTop)

         ' Create a commandbar popup
         _myCommandBarPopup = CType(_myCommandBarToolbar.Controls.Add(Type:=MsoControlType.msoControlPopup), CommandBarPopup)
         _myCommandBarPopup.Caption = "My CommandBarPopup"

         ' Try to retrieve existing command
         Try
            command = _applicationObject.Commands.Item(_addInInstance.ProgID & "." & MY_COMMAND_NAME)
         Catch ex As Exception
         End Try

         ' Create command
         If command Is Nothing Then
            command = _applicationObject.Commands.AddNamedCommand(_addInInstance, MY_COMMAND_NAME, "My command", "My tooltip", True, 59)
         End If

         ' This line hangs VS 2010 Beta 2
         commandBarControlObject = command.AddControl(_myCommandBarPopup.CommandBar)

      Catch ex As Exception
         System.Windows.Forms.MessageBox.Show(ex.ToString)
      End Try

   End Sub

   Public Sub OnDisconnection(ByVal disconnectMode As ext_DisconnectMode, ByRef custom As Array) Implements IDTExtensibility2.OnDisconnection

      Try

         If Not _myCommandBarPopup Is Nothing Then
            _myCommandBarPopup.Delete()
         End If

         If Not _myCommandBarToolbar Is Nothing Then
            _myCommandBarToolbar.Delete()
         End If

      Catch ex As Exception
         System.Windows.Forms.MessageBox.Show(ex.ToString)
      End Try

   End Sub

   Public Sub OnAddInsUpdate(ByRef custom As Array) Implements IDTExtensibility2.OnAddInsUpdate
   End Sub

   Public Sub OnBeginShutdown(ByRef custom As Array) Implements IDTExtensibility2.OnBeginShutdown
   End Sub

End Class

Another critical VS 2010 Beta 2 bug in CommandBars: this one causes VS to evaporate without warning, nothing less

After 3 hours isolating this to the minimal expression (which is tedious if your add-in is has lots of user interface items and debugging a VS 2010 add-in on a virtual machine is painfully slow), here it is:

VS 2010 Beta 2 Bug: deleting a CommandbarPopup unloading add-in causes VS to disappear
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=503093

(the problem happens if the add-in creates two (not just one) CommandBarPopups on a parent CommandBarPopup and tries to delete them when unloaded)

Frustrating new bugs in commandbars of VS 2010 Beta 2

I can’t express how disappointed, frustrated (and more things!) I am with VS 2010 Beta 2 regarding correct behavior of commandbars for add-ins. It seems that once a bug is fixed in a VS 2010 build and an add-in can pass that point, new bugs surface that prevent its correct loading (and I have tested several builds since Beta 1).

Suffice to make you aware of these bugs:

VS 2010 Beta 2 Bug: CommandBarButton of add-in duplicated in new commandbar when already created in another commandbar
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=502289

(For my MZ-Tools add-in with tons of buttons, menus and popups, only that bug of VS 2010 Beta 2 has completely ruined it, not to mention the time it took me to isolate it until I realized what was happening because initially the add-in seemed to hang, due to the number of duplications that were occurring at the glacial speed of VS 2010 debugging an add-in on another VS 2010 instance on a virtual machine)

VS 2010 Beta 2 Bug: CommandBarButton.Style not honored, always Icon + Caption
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=500403

VS 2010 Beta 2 Bug: System.AccessViolationException attempting to create a temporary popup commandbar
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=500408

VS 2010 Beta2: InvalidCastException casting
DirectCast(CommandBarControl, CommandBarPopup).CommandBar.Parent back to CommandBarControl
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=499483

I realize that changing the COM-based commandbars of VS 2008 by new WPF-based commandbars in VS 2010 is a daunting and difficult task, but if the next milestone after Beta 2 is Release Candidate (where only critical bugs are fixed) then Beta 2 should be already free of such nasty bugs. Or there should be a known roadmap of betas / CTPs until RTM so that developers of add-ins can know how many builds they will have to test and rest assured that they will have enough weeks/months to test their add-ins (considering that in Beta 2 they can’t get them even loaded in Visual Studio).

I think that since Microsoft builds and extends Visual Studio with packages (SDK) and not add-ins, commandbars/commands/buttons of packages get better behavior earlier in the development cycle of Visual Studio. But for add-ins (automation) every time that commandbars are changed in some VS release, you have to wait until the RTM to check is some bug that could ruin your add-in is fixed. I experienced the same with VS 2005, which introduced 24-bit bitmaps (rather than the 16-color bitmaps of VS.NET 2003) and a different assembly for Microsoft.VisualStudio.Commandbars.

VS 2010 RC/RTM fixing ” ‘Object must be the same type as the enum’ exception calling CommandBars.Add from an add-in” problem

Microsoft has reported that this problem will be fixed:

 ” ‘Object must be the same type as the enum’ exception calling CommandBars.Add from an add-in”
http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=462766

Notice that the “Embed Interop Types” property seems to have been renamed to “Isolated” in VS 2010 Beta 2, in one of those renaming changes that make things more obscure…

VS 2010 Beta 2 certainly supporting RGB=0,254,0 as background color for command pictures as in previous versions of Visual Studio

FWIW, I have verified that VS 2010 Beta 2 certainly fixes the issue of supporting RGB=0,254,0 for transparent color in 24-bit command pictures that I mentioned here.

So, if your add-in targets VS 2005, 2008 and 2010 you can use that color for the transparent background color of command pictures and they won’t appear with lime green in Beta 2 as it happened in Beta 1 or previous CTPs builds.

If your add-in only targets VS 2010 it is preferable to use the new support for 32-bit command bitmaps. I guess that in a few years Microsoft will only support 32-bit bitmaps in Visual Studio.

VS 2010 and the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface

This is going to be messy, so step by step:

Imagine that you are developing an add-in that uses a temporary user interface (not permanent one), and creates a couple of commandbars of kind “toolbar”, and many commandbars of kind “popup” (with submenus).

Since it uses a temporary user interface, it must delete those commandbars when unloaded.

For that purpose, it must keep a reference to each created commandbar at class level (all this is explained in the article of the previous link).

Since it creates lots of commandbars, rather than keeping each one individually, they are stored in a class-level collection. Of which type?

CommandBars of kind “Toolbar” have the “Microsoft.VisualStudio.CommandBars.CommandBar” type but commandbars of kind “popup” have the Microsoft.VisualStudio.CommandBars.CommandBarPopup” type, so both can’t be stored in the same typed collection, but we would like to use a single collection, not two, or not a collection of System.Object.

Fortunately the Microsoft.VisualStudio.CommandBars.CommandBarPopup type has a CommandBar property, and since the Microsoft.VisualStudio.CommandBars.CommandBar type has a Parent property (typed as System.Object, that should return a Microsoft.VisualStudio.CommandBars.CommandBarPopup actually), you can use a generic typed collection List<Microsoft.VisualStudio.CommandBars.CommandBar>, which stores the CommandBar of toolbars, and the CommandBarPopup.CommandBar of popup commandbars.

When the add-in is unloaded, it iterates the commandbars of the collection to delete them:

  • If the CommandBar.Type is MsoBarType.msoBarTypeNormal, the commandbar is a toolbar and can be deleted calling the Delete method.
  • If the CommandBar.Type is MsoBarType.msoBarTypePopup, the commandbar is a popup so we get its Parent property (which is a System.Object) and we cast it to Microsoft.VisualStudio.CommandBars.CommandBarPopup. Then, we call the Delete method of CommandBarPopup.

(As I mentioned, my MZ-Tools add-ins seems to use every conceivable technique provided by the Microsoft.VisualStudio.CommandBars API)

However, this approach that worked perfectly in VS 2005/2008, doesn’t work in VS 2010. Why?

It happens that VS 2010 uses WPF-based commandbars, but add-ins for VS 2010 still use the Microsoft.VisualStudio.CommandBars Interop (ActiveX) assembly for backwards compatibility.

That interop assembly provides the Microsoft.VisualStudio.CommandBars.CommandBarPopup type, which is actually an interface (although it doesn’t follow the convention for interface names), not a class. The same happens with the CommandBar and CommandBarControl types, which are interfaces, not classes. The CommandBarButton type is a class, though. To complicate things, you have the CommandBars class and the _CommandBars interface… anyway:

In VS 2005/2008, which use native (COM) commandbars, the classes that actually implement those interfaces are native, and from a managed add-in you see them through a System.__ComObject which is the Runtime Callable Wrapper (RCW) that you get when you don’t have an interop assembly. But since they implement the interfaces provided by the Microsoft.VisualStudio.CommandBars assembly, the add-in works fine.

In VS 2010, which uses managed (.NET) WPF commandbars, the classes that actually implement those interfaces are managed and are provided by the Microsoft.VisualStudio.PlatformUI.Automation namespace in the managed (.NET) Microsoft.VisualStudio.Shell.UI.Internal.dll assembly (in one of the VS 2010 folders).

When you create a commandbar popup calling:

Microsoft.VisualStudio.CommandBars.CommandBarControl commandBarControl;
commandBarControl = parentCommandBar.Controls.Add(MsoControlType.msoControlPopup);

you get some class instance that implements the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface. So you can do this cast:

Microsoft.VisualStudio.CommandBars.CommandBarPopup commandBarPopup;
commandBarPopup = (Microsoft.VisualStudio.CommandBars.CommandBarPopup) commandBarControl;

In VS 2010, the actual class that implements that Microsoft.VisualStudio.CommandBars.CommandBarPopup interface is the Microsoft.VisualStudio.PlatformUI.Automation.CommandBarPopup._Marshaler class.

However, when you do this:

commandBarPopup.CommandBar.Parent

in VS 2005/2008 you get an object that implements the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface (as expected), so the approach described in the beginning of this post worked.

In VS 2010 you get an instance of the Microsoft.VisualStudio.PlatformUI.Automation.CommandBarPopup class, not of the Microsoft.VisualStudio.PlatformUI.Automation.CommandBarPopup._Marshaler class that you got when creating it.

It happens that the Microsoft.VisualStudio.PlatformUI.Automation.CommandBarPopup class doesn’t implement the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface. It implements IMarshaledObject<CommandBarPopup> instead. It also has a Marshaller property that would return the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface implementation.

So, the approach that worked with VS 2005/2008 doesn’t work with VS 2010.

Since you can’t cast or get the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface and an add-in targeting VS 2005/2008/2010 with the same dll can’t include a reference to the Microsoft.VisualStudio.Shell.UI.Internal.dll, keeping this approach the only way would be to call its Delete method would be through Reflection.

VS 2010 Beta 2 commandbar-related bugs with fixes on the way

If you are testing your add-in with the recently released VS 2010 Beta 2 chances are that you have noticed that the new WPF-based commandbars, buttons and icons are not yet ready for prime time. It seems that my MZ-Tools add-in uses every possible method, property and technique offered by the Microsoft.VisualStudio.CommandBars API so I am discovering lots of bugs (I have reported 15 so far) since I started testing with Beta 1. I reported these two ones before Beta 2 but the fixed will come in builds after Beta 2:

VS 2010 Beta 2 Bug: System.AccessViolationException attempting to create a temporary popup commandbar
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=500408

VS 2010 Beta 2 Bug: CommandBarButton.Style not honored, always Icon + Caption
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=500403

Debugging add-ins for VS 2010 (Beta 2) created with VS 2008: The debugger’s protocol is incompatible with the debuggee

If you are developing add-ins for in-house use, chances are that you can drive, encourage or even force the whole organization to use the latest version of Visual Studio and therefore your add-in needs to target only that version. But if you are developing add-ins for commercial or open-source purposes, to be used by lots of people, your add-in needs to target several IDE versions (nowadays, at the very least VS 2005, 2008 and 2010). To simplify your life as a developer, you try to keep a single code base and a single binary dll. For example:

or:

  • You create the add-in with VS 2008 (targeting .NET Framewor 2.0 / CLR 2.0) with a single project, with the same references than above, taking care of not referencing EnvDTE90 or VSLangProj90 (which are specific to VS 2008), generate a single DLL, but you create the proper .AddIn file and put in in the proper folder to target VS 2005 too.

Both approaches work. Some people use the first one to ensure 100% that the add-in will work with VS 2005 (since it was created with that IDE and .NET Framework 2.0) although I personally use the second one (I like VS 2008 much more than VS 2005) with no problems so far.

If for some reason the add-in needs to know the IDE version where is hosted or behave differently dynamically depending on the IDE version, it can guess it using DTE.Version or DTE.RegistryRoot, but there is no need to have separate DLLs if both VS 2005 and VS 2008 can use the same CLR 2.0, .NET Framework 2.0 and the same set of references.

Now, VS 2010 comes. So, your natural approach is to keep your existing code, binary dll and make it target VS 2010 adjusting the .AddIn file. Although VS 2010 uses a new CLR (4.0) instead of CLR 2.0, a CLR can load assemblies compiled against the previous version, in the same way that VS.NET 2003 (using CLR 1.1) could use add-ins created with VS.NET 2002 (using CLR 1.0) if the add-in was properly registered.

When you create your add-in with VS 2005 and you want to debug it on VS 2008, you just simply go to the Debug tab of the project properties and set the Start Action to start the devenv.exe of VS 2008, and this works perfectly.

Similarly, when you create your add-in with VS 2008 and you want to debug it on VS 2010, this worked perfectly until VS 2010 Beta 2. With the Beta 2 released two days ago you get the error:

The debugger’s protocol is incompatible with the debuggee

I have confirmed with Microsoft that this is not a bug, it is by design: VS 2008 (using CLR 2.0) or VS 2005 won’t be able to debug CLR 2.0 add-ins hosted in VS 2010 (which uses CLR 4.0). Which is a pity, but this means that your workarounds in the scenarios above are:

  • Use VS 2010 to create a separate binary add-in DLL (CLR 4.0) for VS 2010 only. For me, this is the worst workaround, only considered as a last resort.
  • Use VS 2005/2008 to create a single binary add-in DLL (CLR 2.0) to target VS 2005, 2008 and 2010, giving up debugging on VS 2010, resorting to MessageBoxes in your code to guess what’s going on if you need to debug a problem that can’t be reproduced when the add-in is hosted on VS 2005 or VS 2008.
  • Use VS 2010 to create a single binary add-in DLL (targeting CLR 2.0 /  .NET Framework 2.0, not CLR 4.0 / .NET Framework 4.0) to target VS 2005 / 2008 / 2010. I haven’t test this yet but I hope it works. Even if VS 2010 can’t debug a CLR 2.0 add-in hosted on another VS 2010 instance, you can temporarily switch it to target CLR 4.0 and this should work. Once the problem is debugged and solved, you can switch it back to CLR 2.0. I hope that VS 2010 can also debug a CLR 2.0 add-in hosted on VS 2005 or VS 2008.

Let me know if you have more insights about all this.

The strange case of VS 2010 (Beta 2 /CTPs) closing at startup or failing to create add-ins

As you know if you develop add-ins, Visual Studio 2005 introduced XML-based add-ins that use an .AddIn file that you must place in the proper folder for Visual Studio to show the add-in in the Add-In Manager. Which XML parser Visual Studio uses to parse .AddIn files wouldn’t be of interest if it wasn’t because for some people their Add-In Manager didn’t show XML-based add-ins… and the problem was solved installing MSXML. While I didn’t experience that problem personally, I have experienced a related problem with VS 2010 Beta2 and older CTPs.

The problem was that when I tried to create an add-in with the add-in wizard I got this misleading error message (misleading because I do have the proper language installed).

“An error occurred, and the wizard could not generate the project. Verify that the programming language is properly installed”

And more interestingly, if a place an .AddIn file (even an empty .AddIn file!) in one of the folders that VS scans for .AddIn files, VS 2010 fails to load with the following error message:

“The application cannot start”

and then it closes!

I remembered the problem of the empty Add-In Manager, I thought it could be related, so I verified with the Process Monitor tool that VS was certainly failing to get the MSXML2.Document.6.0 ProgId from the registry just before scanning folders for .AddIn files. I installed MSXML 6.0 and lo and behold, the problem was solved. 

So, I discussed this with several people inside the VS team and they confirmed that VS 2010 will use MSXML 6.0 to parse .AddIn files, etc. (I am not sure if VS 2005/2008 uses that version or older ones). It happens that I was using a virtual machine with Windows XP SP2, and it doesn’t include MSXML 6.0 (it includes MSXML 3.0 SP5), but VS 2010 Beta 2 / CTPs didn’t install it either. When I asked why VS 2010 didn’t install it as a prerequisite, the answer was that VS 2010 will require SP3 of Windows XP (which includes MSXML6), and the final release of VS 2010 will refuse to install on Windows XP SP2, something that the current betas / CTPs don’t enforce yet.

Now I hope that the Visual Studio Team modifies VS 2010 to show better behavior and diagnostics if, for whatever reason, Visual Studio doesn’t find the MSXML parser that it requires. We as developers tend to focus on code paths where things go as expected, and pay less attention to edge cases, but when things go bad, it causes a lot of pain to other people trying to diagnose and solve the problem. Something better than silent errors (empty Add-In Manager), crashes without further information or misleading error messages can be done…