A touchy subject

30 July 2015

Visual Foo, the project that eventually led to Delphi, had one directive; create a development environment more visual than Visual Basic. We took that mandate very seriously. During the height of the project we produced some lovely prototypes that resembled the soon to be discontinued Yahoo Pipes or Microsoft’s long since discontinued Popfly.

What we learned, and I can only assume that Yahoo and Microsoft learned much later, was that such approaches are not better, they are just different. Making something more “visual” doesn’t, by itself, make it better it just makes it more visual.

I put Visual Foo and projects like it on a list of mistakes that every generation will make. I am convinced this model will come back with as much or more fanfare than Pipes and Popfly did and will eventually die just as they did. There are some mistakes that are so appealing they will be made over and over again.

Delphi was a step back from the radical approaches that we experimented with in Visual Foo project. It competed directly with Visual Basic instead of trying to be more visual than Visual Basic. We believed Delphi brought a better compiler, a better language, and a system that was much easier to extend.

I have since developed much more respect for the design of the Visual Basic language than I had at the time. It is still not among my favorite languages but I no longer run away screaming.

When you look closely at Visual Basic you realize that it is neither Visual nor BASIC. It is not really BASIC; it is a long, long way from Dartmouth BASIC or even Microsoft BASIC. It shared some of the characteristics of BASIC and kept some of the syntax but a Visual Basic program looks much different than one in its older cousins. It also isn’t Visual. The only “visual” part of Visual Basic is that it has a form designer. The Visual Basic form designer appeals more to WYSIWYG than visual programming. All the actual programming is done in a text editor, not the form designer. Delphi is the same.

One of the things that made Delphi’s model easier was that you didn’t have to visualize in your head where a thing like the file open dialog was, you could point to it, “It is right there next to the button that opens it.” You can move it around; reach out and touch it so to speak. I never expect this to gain coinage, but this is why I sometimes refer to Delphi’s programming model as tactile programming, not visual programming. Simplicity was helped more by the illusion that you could touch the component, moving it where you want it, not just that you could see it. To keep this illusion it should act as close to a physical object as was reasonable. It needed to feel like you could pick it up.

It is this concept that helps explains why I never created a “gutter” for non-visual components. Any restrictions on a component’s movement breaks the illusion. This also explains why the Data Module looks and acts like a form; it helps reinforce the illusion. If the component can only appear “off screen” or in a restricted area, it no longer feels like a physical thing, the illusion breaks down. Even though non-visual components can get in the way of the forms layout, I felt maintaining the illusion was more important than the irritation the real estate they consume causes.

Making something “visual” helps simplicity when it reinforces the conceptual module of the underlying system. It hurts when it tries to visualize something better left in code.

Simplicity isn't simple

23 July 2015
Button.Caption := 'OK';

This was first line of “Delphi” code ever written. As I mentioned in Properties on purpose Anders wrote this on his whiteboard during one of our earliest design discussion and we needed to get this to work. I already discussed what we added to get this to work, but just as significant is what we removed.

I have participated in, reviewed, read about, or heard about many things purporting to make programming easier. The few that actually did make things easier had one thing in common, they reduced the number of concepts the programmer had to keep in their head.

One of the primary influences to OWL (the ObjectWindows Library), the predecessor to the VCL, was a little known now, but wildly heralded at the time, language called Actor. It was a Smalltalk-like language that made programming Windows a lot easier. However, one way it demonstrated that simplicity in its advertising was to show how you could write a one line editor application. That ad stuck with me, but not for the reason the marketing person who came up with that ad intended. It lead me to this guiding principle,

  • I don’t mind requiring two lines of code to accomplish what could be accomplished in one line if I can understand both lines.

The line of code, through powerful, was completely incomprehensible and couldn’t be reproduced independently without complete understanding of the underlying framework of Actor. Counting lines as a judge of simplicity is a fallacy. It is just an indicator, not an arbiter, of simplicity. The above principle is an informal observation of the fact that sometimes two lines are simpler than one.

Actor made programming in Windows much easier than its predecessors but being able to utter a string hieroglyphics to produce an editor was not the way it did it. It did it by reducing the number of concepts the user had to learn and remember and it did it by reducing the bookkeeping which was, up till then, required.

The first line of “Delphi” code ever written as an appeal to a reduction in concept count. If that was to be written in Turbo Pascal it would have had to look like,

Button^.SetCaption('OK');

The ^ operator, in Pascal, is a pointer dereference operator. First, linguistically, in this context, it is redundant. The member selection operator . is not legal on a pointer so the only reasonable interpretation of Button.SetCaption would be for SetCaption to be a member of the type referenced by the pointer. Second, it requires the understanding when and, when not to, use the ^ operator. We realized that removing the need for the ^ operator allowed us to defer the need for the user to understand pointers. If we allocated the button on the behalf of the user, they wouldn’t need to know New and, later, if we disposed of the memory on their behalf, they wouldn’t need to understand Dispose either. All this was accomplished simply by deleting one character from the line Button^.SetCaption('OK') to form Button.SetCaption('OK'); properties allowed us to further reduce this to Button.Caption := 'OK'.

This actually brought us more in-line with the original Object Pascal as defined by Apple in consultation with Niklaus Wirth which also did not require the ^ in this context.

In the VCL we took nearly every opportunity we could find to reduce the number of concepts the programmer was required to learn and/or maintain in their head when using Delphi as a component user. This showed up in everything from the way menus were defined to how Windows messages were routed.

Reducing concepts can also lead you into trouble. We always kept this maxim in mind (commonly attributed to Albert Einstein):

  • Everything should be made as simple as possible, but not simpler.

I found it ironic that Microsoft chose the name “Einstein” for their, now abandoned, user persona they used for a user who could handle a lot of complexity.

I always interpreted this, when applied to programming, as an appeal against magic, like the Actor incantation for a one line application. The additional requirement of not only simplifying programming for component users, but also for component writers, kept us honest. Any magic we introduced for the component user needed to be fully understood by the component writer; which meant, in our case, magic only postponed, it did not not eliminate, complexity.

The Biggest Bang

17 July 2015

Valentine’s Day, February 14, 1995.

I was sitting the front row at the main hall in the Moscone Center. The lights had just dimmed and a hush fell over the crowd. Several thousand people suddenly in wrapped attention all looking at the stage; flashes of cameras going off, and murmurs of anticipation. Then Anders took the stage and started the demo.

I had never experienced anything like it before or since. I was nervous and excited at the same time. We had built up the expectations of everyone, I was hoping we didn’t disappoint them.

Apparently we didn’t. It seemed that after every feature Anders showed people cheered and when Anders showed that Delphi itself was a Delphi app we received a standing ovation. I knew we had done something remarkable; something I will always be proud of.

From the yes! we received to that amazing night was about 2 years. That was the longest I had every spent working on one release and over a year longer than we had originally planned but we were finished and it felt very, very good.

Well, almost finished. The first versions of Delphi were not due to hit the streets until March 1st, 2 weeks after the 14th and we were in final stages of signing-off the product but we were not finished just yet. There were a couple of “stop ship” bugs that needed to be fixed and, as I remember it, Marc Cousineau was the last man standing. He missed that night. We missed him there but he took the last few bullets so the rest of us could enjoy ourselves. Thanks Marc!

How did we know the features we chose were the right ones? How did we know we were done?

The biggest challenge of building a product is often not what to do, but figuring out what you don’t have to do. Of all the features you can build, which should you build? Which can wait and which cannot? Even in `95, 6 months before the earth shake that was Windows ‘95, the Windows API was large and complicated. What parts of the API should we wrap, which should we ignore?

What we did was an application of the 80/20 rule, or as I learned learned later was called the Pareto principle. Applying this to something like the Windows API means that roughly only 20% of the API is used over 80% of the time. This meant we didn’t have to wrap everything, we only had to figure out that 20%. Or, as Anders likes to say, we needed to go for the biggest bang for the buck.

But what about the other 80% of the API that is used 20% of the time? We knew two things. First, we would never have time to wrap it all. We didn’t have the people and we couldn’t keep up producing wrappers for things from a company as large an prolific as Microsoft. Second, no application could be written that was worth much that didn’t, in some way, use part of that 80%. This lead to the principles,

  • Make simple things simple.
  • Make the hard things possible.

The most straightforward example of this principle in action is the Graphics unit with TCanvas and its related classes. All of the Graphics classes that wrap an underlying Windows API handle expose that handle through an aptly named Handle property. This was unique at the time for a high-level encapsulation. The user is allowed to do just about anything with the handle and the underlying object would adjust accordingly. Anders and I spend a considerable amount of time enabling this; making the easy things easy, and hard things possible. TCanvas made using DCs easy, exposing the underlying handle we used made calling APIs we didn’t wrap possible. You can see this principle applied throughout the VCL, the underlying handles are exposed to make the hard things possible, even though doing so wasn’t exactly easy.

As I mentioned previously, we intentionally made the VCL extensible. We didn’t have the people to wrap everything and wanted our users to be able to build or buy the components they needed to create their applications. Allowing components to be written in the same language they used to write the rest of the application means we lowered the bar considerably for application specific encapsulations.

We began Delphi in early 1993 and we were locked and loaded for a release sometime in that winter. In fact, it didn’t take us that long to get something that looked substantially like Delphi up and running. I remember a demo we did in July of 1993 to a group of engineers showing them what we had at the time. It included the component palette, form designer, and a rudimentary editor and even included what we later called two-way editing.

The editor was actually just a TEdit so very rudimentary! By July it wasn’t just me and Anders anymore. We had decided to cancel work on what was going to be Borland Pascal 8.0 and focus completely on Delphi. Allen Bauer and Alex Shtaygrud began working on the debugger and editor (explaining why we were using TEdit) and Dave Scofield began making my chicken scratches of an IDE look presentable. Ironic that a developer that spent most of his career building UI libraries and frameworks cannot create a decent looking UI to save his life.

After the meeting, I remember one of the engineers saying we should ship it and ship it now! We were encouraged but knew we had a lot of work to get us to a shippable state.

Then came Visual Basic 3.0 and with it the 80% changed dramatically.

Will the true component please stand up

13 July 2015

There are only four words in computer science and component is one of them.

Components

When we used the term component in VCL we meant a class that was primarily defined by its properties, its methods, and its events. I believe it was Zack Urlocker I heard stated it this clearly first but once I heard it I knew it was true. That is exactly what we meant by component.

Properties

The state of the component is surfaced exclusively through its properties. The some of the state of the properties should reflect the complete state of the component (that is, little to no hidden state). The properties can be changed directly and the state of the component should immediately reflect that state or it should throw an exception, rejecting the change; however, it should only do the latter infrequently. Ideally the type of the property should be the only filter on the values of the property.

All properties should be independent, or as independant as possible. Changing a property should rarely affect the value of another property. Sometimes it is unavoidable but those cases should be minimized.

This principle is enforced by the serialization system. A Delphi form is the serialized state of a set of components, serializing the values of their properties. They are deserialized by constructing an instance and setting the properties, one by one, from the form file. Any state not surfaced as a property will not be serialized. Coupling of property values can cause the deserialized version of the component to be different from the originally serialized version.

Note this is not literally true as components, such as an image, have always been able to store binary data not surfaced in a property but I am not one to let the facts get in the way of a good story.

By state above, I mean specifically the initalization state, or the configuration state, of the component; not the running state. Components can and should use encapsulation, just like any other well designed object.

Events

An event is just a property with a method pointer type. It is called out independently because the way the user interact with it is much different that other properties, since it is how the component signals the user of important events, but, deep down, it is just a property. There are special principles around events that make them easier to use, however.

Events should be contract-less. This is hyperbole but, as demonstrated above, I am a big fan of hyperbole. What I mean is that, in an event handler, the user should be able to do anything they want; change focus, terminate the application, destroy the component sending the event, anything. Any of these should never raise an exception or cause unexpected or undefined behavior. The event should also never expect the user to do anything. That is, an empty event handler should always be legal. Logically, this means that event method pointers should always be procedures, not functions (functions require a result, violating this principle). An empty event handler must be treated semantically identically to having no event handler assigned. There are always going to be exceptions to these general principles (OnFilter being one obvious example) but any event should try to adhere to these general principles.

This is the place where the contrast between a framework and a component library is the most stark. A framework uses its virtual methods to ask questions the user is expected to supply answers to, such as, “What color would you like your hints today?”. A component, on the other hand, just tells you about interesting things you might want to know about such as, “By the way, the user pressed the letter A. Just thought you would like to know.” What you do with that information is strictly up to you.

Methods

There is not much to say about methods, methods are verbs that tell the component to do something. There is not much restriction on what that something is.

Learning after the fact

Properties should be changeable

Sometimes you can really only learn what assumptions you are making when presented with a contrasting example. One assumption I had that I never wrote down or even consciously thought about was that the state of a component after a property change should be the same as had the component been born with that state. It is a bit subtle but important. For example, there should be no one-way trap-doors in a component’s state. If you add a border to a control, should just as easily remove it and it should appear as if it never had one. This was enforced by the form designer. The form designer creates real instances of the controls you are editing, not fakes. If you violate this principle the control will have weird behavior in the form designer.

After leaving Borland I went to work at Microsoft on Avalon, later called the WPF (Windows Presentation Foundation) and even more recently, simply XAML. In contrast to the VCL, once a property is set in XAML, it is rarely ever changed. Once the view was created and displayed on the screen (that is, added to the visual tree), properties would often become immutable. You can make changes in XAML that cannot be made at runtime. This made the framework perform well in a lot of situation but it made writing a designer for it, my job, hard. I stubbornly reported each one of these immutable properties as bugs and continued on my merry way writing the designer. It wasn’t until later, when all my bugs were marked “working as intended” that I realized my mistake. They rightly rejected my attempt to slow WPF down to make my life easier. XAML is not the VCL and I shouldn’t have treated it as if was.

Delphi events are single-plex

Events are commonplace now and are seen in just about every user interface library and framework. It is only in Delphi, however, they are single-plex. That is, in Delphi an event has one and only one handler. That is because the type of the event is a method pointer, a pointer to one method.

After using multiplex events in several frameworks, I don’t see them as an improvement. In Delphi, at least in 32-bit Delphi, a method pointer is 8 bytes, two pointers. It is a value, not an object. An just like other values is immutable. This allows the event to just be a property, as noted above. In systems with multi-plex events, they are treated quite differently from properties. In .NET you have the add_Event and remove_Event methods (which is what C#’s += and -= is translated into calling). In JavaScript you have the addEventHandler or on or any number of patterns that mean the same thing. Each event must be able to handle a list of handlers and be able to dispatch to that list. In JavaScript, for example, there is an EventEmitter class whose sole job is to track these values and make emitting events tolerable. One thing all these ignore is that having more than one handler for an event is rare to the point of non-existence. All add significant overhead to the simple event handler just to handle this one case, dispatching to multiple handlers. There are better ways to handle multi-plexing events than to make every event a boat anchor. Single-plex events have their own problems but I still believe multi-plex events are not worth their cost.

A component of the story

10 July 2015

The VCL, the Visual Component Library, is not an object oriented framework. It never has been. In fact, when I designed it, I made sure it wasn’t because I had lost my taste for them having written two, Turbo Vision and OWL. I loved Turbo Vision but, as Jeff Duntemann told us, we put the front door on the second floor. You had to understand everything before you could understand anything. It was great at what it did but if you needed to learn and buy into every assumption we made or it would fight you the whole way.

In many ways, OWL (ObjectWindows Library) was even worse. We tried to learn from our mistakes with Turbo Vision but we made too many new ones. OWL was intended as a thin wrapper on the Windows API that would simplify and highlight the underlying object oriented nature of Windows with language support. Unfortunately, approached that way, Windows is a terrible example of object-oriented design and all we ended up doing was making the flaws more apparent. The C++ version of OWL did a better job of hiding the flaws but, by the time Borland started work on that, I was already in the middle of the VCL.

After two frameworks I saw the flaw in the very nature of frameworks. A framework is like a fill-in-the-blanks form with a lot of little places where the framework designer allows the user to fill in code. If the framework resembles the finished application you want, you are golden. If it doesn’t, pick another framework. It was that all or nothing, the framework is king, model that I found distasteful. The user of a framework extends the framework through inheritance and the blanks they fill in are virtual methods. If you think of the framework as a tree data structure, users are extending at the leaves, at the bottom of the tree. The framework is on top, the user extends at the bottom at whatever leaves the framework designer deemed worthy.

With the VCL we wanted to start fresh and that meant I was not doing a framework. I didn’t want give the user a set of blanks to fill in, but, rather, give them a toolbox full of useful tools and supplies they could fit together or ignore completely as they saw fit. I wanted the user writing the application to be in charge, not me. I wanted the user on top of the tree and the components at the leaves.

Some principles of the design of the VCL I came up with are,

  • The users should feel they are in control at all times.
  • Components should appear independent and self-contained.
  • Inheritance should not be required to construct an application.

To do this, in my mind I separated users into two camps; component writers and component users. Often they were the same person but it was a useful separation. Component users should be able to be successful constructing a full application with just the components they have and writing code to stitch them together. They should never have to know or care about object-oriented concepts like encapsulation, inheritance, or polymorphism.

The component writer, however, would care very deeply about object-oriented programming. A component, after all, is a class that inherits from one of the fundamental base classes. But once the component is complete, the inheritance it used for implementation should be a bit of interesting trivia, not the core of its being. From my perspective, the user of a component shouldn’t need to know or care what its ancestor is, just what properties, methods and events it has. The fact it had an OnMouseDown event, just like the OnMouseDown event of some other component was a happy accident not because they both inherited from TControl. In reality, they both needed to inherit from TControl but that should only be important to the component writer, not the component user. The Delphi 1.0 documentation reflected this philosophy; something it lost over time, unfortunately.

The name of the VCL, Visual Component Library, is no accident. It is a library of components not a framework. It contains a framework, a framework to enable writing components, but it is not a framework itself.

The focus on components also made the third-party ecosystem around Delphi very clear. The plan was for users to be able to purchase third-party sets of components that would fit into Delphi like they were natives; which, in a way, they were. There is little difference between a first- and third-party component in Delphi as they are both built with the same tools. This was intentional from the very start and we believed would be the key to Delphi’s success.

Older posts

See archive for older posts.