BfME & Context-Sensitive Interfaces

[Acumen]

I didn't want to clutter up Jason's excellent report, so I've split this to a new topic. My intention here is to describe the theory of noun-verb interfaces, and see why we should and shouldn't consider using them in 0 A.D.

Secondly, I haven't regrettably played BfME yet (though I have studied all the Inside the Battle movies, and any screenshots/previews/reviews I can get my hands on, rather like a deluded fan boy ), so if I've made errors of interpretation, please let me know.

1. "The Legacy"

As time has gone on, RTSes have become progressively more complex. There was a time when a simple Windows-like point-'n-click interface was adequate. Clicked a unit? He's selected. Clicked empty ground? He'll move there. Clicked an enemy? He'll attack him. No extra buttons, no clutter, no awkwardness. Just one click. Intuitive to anyone. The Marketeers rejoiced.

As time has gone on, units have needed to do more, and players have come to expect it. They've gotta be able to garrison, repair, heal, patrol, escort, cast fireballs, deploy into more powerful forms, and so on.

This has made point 'n click insufficient, since more than one command could be linked to a context. When the player commands a unit to perform an action on a ship, does he want him to board it? Repair it? Scuttle it? When the player right-clicks an enemy building, does he want to attack it or capture it?

Typically to fulfil these extra commands, these are added as extra buttons in the bottom bar of the interface. The player drags the mouse over to the button (eg Garrison), clicks it, and drags it back to wherever he wanted to perform the action, and apply that command to the target. I call this a "verb-noun" interface.

This is slow, inefficient, and counter-intuitive, since typically the player was probably hovering his mouse around the target in the first place.

The extra commands are also taking up space on the screen, even when not in use, when the player's focus really should be on the game world itself.

This process can be sped up by using hotkeys, of course (press key to change command, click target), but that creates a barrier to new players that have to memorise hundreds of key-presses in order to have a chance.

(Aside: BfME seems to use precisely this method to control the special abilities of its units, which looking at their movies seems to be just as awkward as ever -- though they did move those buttons closer to the centre of the screen where the player is likely to need them, cutting the travel time in half. Also, by drastically playing down the economic aspects of an RTS, the player has more time to micro these individuals heroes and their abilities.

Given the choice, I would have these units use their combat abilities automatically, perhaps linked to power bars if too powerful, a la AoM, so the player can focus more of his attention on the macro coordination of troops, instead of fiddling with the interface in the heat of battle.)

Anyway, the alternative is a context-sensitive interface.

2. "What the Sam Hill is a context-sensitive interface?"

It's basically the opposite of the above, a "noun-verb" interface. The player clicks the target, and when more than one option is available, these pop up in the area around it. He then only has to move the mouse a very short distance, pick from the available choices, and the command is fulfiled.

To show just how sad I am here's an example from The Sims 2: The player clicks a Sim's fridge. Depending on the Cooking skill of the active Sim, there are options like "Cook Cheese Toastie", "Cook Pork Chops", "Cook Lobster Thermidor". There are also generic commands like "Restock Fridge" and "Juggle Bottles".

The implementation can differ drastically from game-to-game, but the general idea is rather like the context menu you get when you right-click in Windows. The computer works out the situation, and depending on the context you have options for Properties, Cut, Paste, Select All, etc. If a command isn't suitable to this context, it doesn't appear.

This has various advantages:

* The player only sees what we needs to see:

Commands only appear when he needs them, and when they'll work on this target. No clutter.

* Easy-to-use:

It's still a mouse-driven, intuitive interface.

* Rapid reaction:

Travel time is far less.

3. "This doesn't look like Nirvana to me."

Of course, this method has disadvantages too:

* It works best on immobile objects:

The ring of context-sensitive commands needs to stay in orbit around the target. If the target moves around, the commands would move with him, making it difficult to click on them. Thus, this is an interface only really viable for large, non-moving objects like buildings.

* It works best when there's only a few commands available:

The more commands pop up, the more room they take up on the screen (generally an orb of buttons shouldn't take up more room than the target itself), the more overwhelming the choices, and the longer it takes for the player to find his desired option.

It works well in BfME, but they also removed a heck of a lot of options that the player normally has. There are only a handful of unit types. There are few techs to research. Building doesn't require units or placement of a footprint. AFAIK a number of commands like rally points and garrisoning and town bells, have been removed.

Another option, when there are too many of them to take in, is to package groups of commands under certain categories (rather like the tab system already in place).

So to use The Sims 2 fridge example above, there might be options "Make Dinner ..." and "Make Dessert ...". Pick the dessert option, and the ring is repopulated with sweet and tasty choices like "Gelatin" and "Baked Alaska".

But again, tabbing commands makes it require one more click in order to get to them, slowing down interaction rate.

* Sometimes, you *need* to pick the verb first:

A good example is standard construction of buildings. In BfME, they simplified this by putting plots on the ground that were a fixed distance from each other, and these were the only places where the player could build. The player clicks the plot, chooses a building, a peon appears out of nowhere nearby, and it starts constructing.

We, however, don't have that liberty. Players could build anywhere that there's a flat surface. They can be of varying size. And the command to build is given to a unit (which as explained above, could move around).

Take the Mobile Construction Vehicle from the C&C games. This slow-moving unit had to be moved to a particular spot, come to a halt, then you click on it to see if you have a deploy option (which would cause the MCV to turn into a Construction Yard building). If you didn't, then something must be in the way of where its footprint would be, and you'd have to move it around blindly until you found a decent spot.

That's pretty much how free placement of buildings would have to work in a context-sensitive interface.

4. "In conclusion."

Context-sensitive interfaces are still something I'm hoping to look into. There's definitely potential there for increased ease of use, assuming they can function optimally with the wide variety of commands that are required. (Perhaps a hybrid interface, with "passive" commands (those that don't require a target, like Town Bell) on the bar, and "active" commands (those that do require a target like Garrison or Repair) handled by a context menu.

Currently the game GUI isn't capable of supporting such features; it's a 2D interface applied over a 3D world, with no means of attaching a control that retains a position relative to that world, so this concept isn't something that requires any consideration at the moment.

However, in time we will need to start implementing controls with an awareness of their environment (such as hitpoint bars attached underneath units), and I hope that functionality will provide opportunity to experiment with this intriguing approach to user input.

[Wijitmaker]

Aside: BfME seems to use precisely this method to control the special abilities of its units...

It does, and it is annoying. It's slow and cumbersome. If you wanted to use your heroes to thier upmost potential, you'd have to use key commands because the UI just isn't fast enough.

One thing they did do to speed things up was to have the hero unit's portraits in the bottom of the window. It was sometimes hard to find your hero on screen when its surrounded by simliar looking units. So all you have to do is select that hero icon and it selects him.

Also, by drastically playing down the economic aspects of an RTS, the player has more time to micro these individuals heroes and their abilities

Yeah it would have been impossible to manage the heroes if they had AoE type ecconomics. It would be just too much management.

The player clicks the target, and when more than one option is available, these pop up in the area around it. He then only has to move the mouse a very short distance, pick from the available choices, and the command is fulfiled.

I personally love this feature in BfME. It makes the game so much easier to control. When we were first chatting by email (the convo the evolved into the alternative UI discussion) I think I brought this idea up (around the middle of september I think?).

---------------------- Random Thoughts ----------------------

Biggest concern, I don't know if its feasible to do this? I remember you had to ask Gustav about the ability to anchor sprites to a 3d object in the world and have it interactive? Maybe that has been resolved. If it would be to complicated to impliment an anchor, what if we just had a menu that could be popped up by a simple keyboard command like the spacebar? It would always pop up smack-dab-in the middle of the screen. Make your selection and it would disappear, or hit space bar again to make it go away.

Icons around small things.. It works great around large objects like buildings (as you mentioned). Maybe even work on BfME units that aren't acting as just a single unit, but a battalion of units. But, for a single unit? Not sure how that would work.

Icon placement and visibilty blocking... Not sure how well this would work cause this ring of icons would be blocking out the objects behind it. Alternatives? Maybe use some sort of opacity? Simple sihlouette icons would probably be usefull in this case.

Should we duplicate the icons in the ring around the entity and in the UI down below?

I agree with your tab pro/con thoughts. However I think some organization would be better than just one big ring. If we did do the tab idea, I envision we would need some sort of 'back' button in the 2nd layer to get back to the 1st layer. Maybe at the 12:00 position?

About the building... Not sure if you would have to do it like the C&C mobile construction vehicle... what if you did it like this. Select the citizen soldier (primary ring pops up), select the build icon (secondary ring pops up), select the econ button (tertiary ring pops up), -oops you didn't want an econ building-, click the back icon in the 12:00 position (secondary ring pops up), select the millitary button (new tertiary ring pops up), select the barracks, transparent version of the barracks appears, move the building around the map till the base is no longer red, click to place, hold and move your cursor to forms a vector that tells you how the direction is to be placed (ala Generals). Foundation appears...

Then the dude walkes over and starts building.

Good stuff though, I like the context sensitive UI. I think it would be pretty powerfull for our gamers. It allows the mouse users to get one step closer to the key-command users.

[Enarwaen]

hey guys - sorry to 'invade' your topic here

a game that has a quite nice context-sensitive interface is 'Neverwinter Nights' from Bioware (a D&D fantasy rpg) - although in hectic situations it becomes (atleast to me) almost unbear/playable. good thing the game comes with a possibility to pause (pressing space) anytime so you can issue (or better queue up) commands.

i agree that a symbiosis between conventional and context-sensitive interfaces is a good way to go ... as is the concept of limitting the context-sensitive interaction more to the 'static' and 'slower' game items (i.e. buildings, resource gathering ...)

anyway - thanks for the interesting read(s) in here

[Dnas]

Maybe have the context sensitivity as an option so that players used to the tradition RTS UI won't have headaches and players who want something new will be able to have the new UI.

I like the idea of implementing it into 0AD.

[Acumen]

One thing they did do to speed things up was to have the hero unit's portraits in the bottom of the window. It was sometimes hard to find your hero on screen when its surrounded by simliar looking units. So all you have to do is select that hero icon and it selects him.

Also makes it easier to keep track on their progress (it shows their health, level, etc).

Should be easy enough to do, if you want it. Depends on how much of a role you intend for heroes to play (eg integral to the plots of campaigns, required to complete objectives, etc, as in AoM and Warcraft 3).

In WC3, they similarly stack Hero portraits against the side of the screen, where you can click to select/centre them. In AoM, they had one button that cycled between however many Heroes you had on the map. Current game rules state that in MP, we'd only have 1 Hero at a time, so it's less of an issue, but the player could be given anybody in a campaign.

If it would be to complicated to impliment an anchor, what if we just had a menu that could be popped up by a simple keyboard command like the spacebar?

Right now anchoring isn't implemented, though as mentioned we'd need some kind of anchoring capability at some point to handle things like health bars (unless they're hardcoded into the entity system, like selection circles are).

If they don't need to retain a position relative to the target, then I could start experimenting with them now.

However, without that relative positioning, you lose a lot of the benefits of this scheme. Travel distance is increased if you're, say, clicking a unit in the corner of the screen. Usually the icons only appear when you hover the mouse over the unit, but here they'd be visible all the time unless disabled (though translucency helps), that kind of thing.

Still, would give an opportunity to at least set some ground work in playing around with it.

About the building... Not sure if you would have to do it like the C&C mobile construction vehicle...

Sorry, I didn't explain that very well. What I meant was that if we did it like: click builder unit; click ground; building options appear around ground; pick a footprint ... then we'd get the issues I mentioned. Placing buildings of varying sizes is an example of when we can't get around having to use a verb-noun approach.

As for the approach you suggested, yes, that'd work well in theory. The things to consider have already been mentioned ... a] Whether it'd be awkward to activate that interface around small, mobile units; b] Whether it'd take longer to navigate through multiple rings of options to get your choice, versus the traditional "click in the corner" method. As pointed out, context-based games with lots of options tend to need a pause option (as in Neverwinter Nights and TS2) as the complexity gets out of hand. Though with only a few choices, it's very efficient.

Anyway, thanks for your comments, guys. If I get some time I'll try to play around with it in our GUI and see what I can work out.

[Calefaction]

Wanted to quickly add a side thought:

The ability to 'anchor' a control to a unit in the world should not just be a 'nice thing to have', it should be (and in my mind must be) a requirement of the GUI system.

[Acumen]

Agreed.

[Ykkrosh]

Attaching things to entities should be easy, by just storing a reference to the entity inside the object (then transforming its position to the GUI coordinate system and adding the normal 'size' attribute every frame), but I don't think the GUI is currently dynamic enough to do things like create a new health-bar object every time you select a unit. But that's mainly a problem in the JS interface rather than the C++ code, and so should be reasonably easy to solve.

Health-bars would probably have to be a new type of GUI object to avoid running hundreds of JS "healthbar.value = entity.health" commands every frame, and then there's not much point even using the GUI for them, so it's probably not a good example; but BfME-style buttons that are attached to buildings should work fine with a simple "x=CreateNewGUIObject("button"); x.style="button_build_building"; x.size="-16 -16 16 16"; x.attach_to_entity=somebuilding; ...")