Why searching is so difficult to learn

04. mai 2009Jørgen Dalen

Research has shown little change in search behavior over the last years. Search engines have improved greatly, people haven’t. Why? Is it because users have a poor understanding of search systems, or is it because every search act is unique so there is little to learn from previous experience?

It is argued that most search systems by their design lead to learning by trial and error rather than learning by insight which is more effective. The objective of this article is to show how learning theory can be used to understand search behavior, and to give examples on how search interfaces can be designed to facilitate learning by insight. This is becoming increasingly important as volumes of information grow and finding the correct information becomes more critical.

Background

Product development is often driven by the need for a more comfortable user experience, for example cars having window wipers that turn on automatically when it rains. It is a paradox that products themselves get more complex because we want them to be easier to use. The result can sometimes be the opposite of what was intended if the designer fails to hide the functions of the product that are irrelevant to the user.

Is it important to understand how things work?

You don’t have to know anything about the profession of editing to enjoy a good movie. Sometimes it can be more entertaining not to understand how things work, for instance when you watch a magician performing a trick. In the examples above the user plays a passive role (even if the magician can fool you into believing you have influence on the outcome). In interactive systems such as computer applications and Internet websites, users must have an understanding of how the system works to be able to use it effectively, at least on a functional level. Search engines are indeed interactive systems, but for most of us they appear as a ”black box” in the sense that we only have rudimentary knowledge of what happens inside it. Is knowledge of the internal properties of search systems important to the users?

Learning by trial and error

Not knowing how things work often means people have to learn by trial and error. In the context of searching, the trial and error process starts off with the user typing more or less random search strings. Some of them give good results, some not. Successful behavior is reinforced and tends to get repeated. What is learned is a response tendency. According to behaviorists, genuine understanding of the system is not necessary for learning to happen; it is enough for the user to know that certain responses tend to be rewarded. The hallmark of learning by trial and error is that the learning curve shows slow and incremental improvement, indeed very similar to the general development in search behavior the last years. However, let us not jump to conclusions.

Learning by insight

Insight is a central part of Gestalt theory and has been defined as “learning that occurs rapidly, is remembered for a considerable time and transfers readily to situations related to the one in which the insightful learning took place”. Having insight means that a user has a genuine understanding of how a system works.

A now classical study by Tolman and Honzik pinpointed the difference between the two kinds of learning. In the experiment two groups of rats were placed in the middle of a maze. One group of rats was from the start given reward each time they made a correct turn, until they found their way out. Another group had to stay in the maze for several days without reward. When reinforcement was introduced after several days, this group showed a rapid rise in performance, and soon outperformed the rats that had been given reward from the start. The fist group of rats had learned a set of individual responses, while the latter had learned “the lay of the land” or cognitive map as Tolman coined it, which in the end proved to be the most effective.

There are several reasons why the cognitive behaviorism of Tolman is useful for understanding search behavior. It focuses on goal-directed behavior, which searching usually is, and tries to explain complex mental processes through the observation of behavior. Searching is indeed a complex problem solving activity, and at least parts of this activity is possible to observe.

Properties that facilitate learning

There are several factors that make learning by insight difficult in the context of search. First, the users must know something about the information that is being searched, and not only the properties of the search engine itself. This means that each search act has the potential of being a “new maze” for the user. Secondly, the users have to relate to many different search engines, which make knowledge transfer even more difficult.

Since insight is so important to gain, the key question is: How can search systems be designed to facilitate learning by insight? This article will discuss three properties of search systems that are all central to this question, namely transparency, convention and feedback.

Transparency

A real challenge when designing search interfaces is to make transparent the relevant aspects of the system while hiding complexities that is of no use to the user. It does not facilitate learning to tell the user how fast the search engine managed to retrieve the results, or what kind of algorithm the system used to correct spelling errors. Other aspects of search engines are more relevant, but can be hard to explain to the user, for instance the difference between keyword search (works well with Google) and semantic search (does not work well with Google). How do you make that difference clear for the users? Some vendors have solved the problem by designing systems that perform keyword search when the user types fewer words and semantic search when longer sentences are used as search strings, without letting the user know the difference. The strategy of allowing several search behaviors without showing the technical solution that deals with it can often be a good solution.

As mentioned earlier, understanding the information that is searched is a very important part of the search experience. While technical aspects of the search engine can be hidden, properties of the “information space” that is searched should not. It is a paradox that the difficulty of describing the information space for the user has nothing to do with the size of it. Google for instance has an information space that is easy to understand, because it contains “everything” (relative to site-specific information that is). Searching single websites is more difficult than searching the entire web because the user has a poorer understanding of the extent of the information space.

Convention

Conventions are important to both kinds of learning. In Gestalt theory, conventions are related to the concept of “transposition”, which means that a principle that is learned in one situation works in other situations as well. In behaviorism, conventions are linked to “generalization” which means that response tendencies learned in one situation will be rewarded in other situations as well.

Learning transfer from one search system to the other would be easier if they performed in a similar way. This is very important in the context of the web, where users in general show resistance to learn new principles. Today there is practically no Internet search system that completely shares conventions with another system.

Feedback

Precise feedback can change behavior a lot, but does not necessarily lead to insight. A problem with search systems in general is that the user gets too much, too little or even ambiguous feedback. The number of hits tends to be higher than the user wants; there is simply too much feedback. Ambiguous feedback is a risk because a search result can contain both relevant and irrelevant results, which means that the reinforcing mechanisms important for learning are less clear to the user. On the other hand, the fear of not getting any feedback can lead to the behavior of using broader terms than necessary when searching. Relevant and informative feedback when the user gets 0 hits is very important to the learning process. It is also of great help to display the taxonomy of the site during search, offering to the user more specific and precise terms. 

Feedback can also be given for a larger set of search entries, and not only to single responses. If there is consistency in the way the user searches, the probability of giving correct feedback will increase.

Conclusion: Lowering the walls of the maze

Learning theory in general and Tolman’s studies in particular can be effective tools to describe search behavior. It is also suggested that learning by insight is possible in the context of search, especially if the properties of transparency, convention and feedback are added to the user experience.