Notes

How to argue that a proposed design works?

The ultimate evidence is, of course, to show the actual device working. But often it requires the (risky) expenditure of scarce resources to physically build a designed system. This means that a designer or design team must convince the gatekeeper for those resources (a manager or project lead, for instance) that the design that currently only exists on paper is likely to actually function and perfom as intended.

In a student project, the situation is slightly different. Here, it will be the students themselves that will build their design, not uncommonly at their own expense. So why do we still ask them to convince their teacher that going ahead is the right decision? In this situation, the risk is not the financial cost of a failed prototype but the lost time and opportunity in the course. Failure during a course will lead to less learning, more effort on the part of the teachers, and at worst a need to take the course again for the student.

So how does arguing for a design ‘on paper’ work? First of all, before we can get to whether the argument is convincing, for it to be sound, it needs to be clear what is being claimed. This means that it must be clearly stated what the intended function is, why it’s valuable or desirable, what the requirements and restrictions are, and also what performance criteria should be used to judge the design.

Here, we get to three necessary claims:

that it works (what does it do?)
that it works well (what does good performance look like?)
that it’s the best you can do (are there no obvious and better alternatives?)

The first two of these seem at first glance to be relatively straightforward. Quantitative modelling, physical reasoning, and calculating expected values for the product’s features and performance seems what’s called for. But how do you argue the third point? How do you convince people that the proposed means to fulfill some function are the right, appropriate, or even the best means?

In my experience the answer given to this question is often a variation on “good, structured design process”. I agree that a ‘good’ process is the means to produce this argument, but it isn’t itself the reason. A rigorous process leads to considered alternatives, and it is comparison to alternatives that provides the persuasive force to accept this particular design as the preferable one. In fact, this is the only way, it seems to me, to argue for the appropriateness of a certain design to attain a certain goal. It is easier to produce appropriate alternatives through a structured, disciplined design approach, but how the alternatives are generated does not matter in the final argument on which design to accept.

The question of concept selection is distinct from the question of optimization (the second question of the three above). A clear argument about what performance criteria the design was optimized for, and that it is indeed optimized for these, only supports the claim that a local optimum has been achieved in the design. It cannot support the claim that other local optima (the best versions of designs that are fundamentally conceptually different) aren’t even higher.

This leads to the burden of proof for alternative concepts: as a designer or design team, you need to convince me that each of your concepts has been optimized towards its maximum performance, that you’ve reached the peak of the local optimum. Only after this has been established, can the concept support the further claim that another concept –with a higher expected value or performance– is preferable. For this you also need to establish that none of your concepts’ expected performance is above their achievable level, for example because an unsolved problem still exists whose resolution would detract from the quality of the concept.

Underlying a (small) set of concepts that are established as embodiments of local optima in performance there needs to be a further argument: that the concepts that were developed into complete (if rough, or abstract) design proposals represent the most promising conceptual possiblities. This requires some overview or mapping of all possible conceptual approaches to the design problem.

This entire edifice of design justification needs to be clearly presented, understandable, and accessible to a judge of a design proposal. They need to be able to go through each part and decide whether they are convinced of each part.

Asking Why

Design teachers continually ask their students: why? This is frustrating for the student and in the end, ineffective. Daniel Dennet’s two versions of “why?” may help us think this through.

Students interpret this question, I think, as “how come?” In any case, that’s often how they answer it. They start telling us about all the steps in their process, the changes, developments, and other design moves they made that culminated (for the time being) in this particular feature.

The teacher, I think, is interested in “what for?” What is the value or function of this feature? What is the effect? But often, there probably is no intended effect. This is just the first shape that came to mind, or the dimension that fit without causing any explicit problems.

Come to think of it, the student may very well understand that the teacher is asking “why?” in the sense of “what for?”, but when they don’t have an answer, they just start describing their “how come” origins.

And, in fact, it doesn’t really matter whether there is an intended effect to answer the teacher’s “why” question. The answer might be, no reason — yet. Because that’s why “why?” is an interesting and potentially productive question: what might or could the effect of this feature, nut, bolt, angle, or dimension be?

A well considered design is exactly that: rigorously considered. This means that for every ‘independent variable’, for every feature under the designers control, and thus everything the designer is forced to make a choice about, it has been considered what the effect is, what effects could be produced by varying this variable, whether these are positive and could be further strengthened or whether these are negative and could be minimized or compensated for somehow.

Grades

It is taken for granted that students taking a course will receive grades. But when we give a course or workshop to colleagues, or when we follow professional training as part of our jobs, nobody even thinks of handing out grades. We’d actively protest, I believe. This would be silly. We’re adults, thank you very much!

We don’t mind being assessed. In fact, many participants in courses, workshops, and training programs welcome feedback, positive and negative (constructive). And we expect to be tested. But we expect the result to simply be pass/fail. Why on Earth should we get grades on a scale of 1-10?

But then, what’s so different with students?!

Ontwerptheorie, ontwerpkritiek

Het is al vaak opgemerkt, maar het blijft een ding dat er zo weinig van ontwerptheorie door ontwerpers in de praktijk als relevant ervaren wordt. Een manier om dit voor nieuw werk te voorkomen, kan zijn om iets aan te pakken dat daadwerkelijk als een probleem of open vraag beleefd wordt.

Ik denk dat de vraag wat het betekent om universitair, academisch bezig te zijn als ontwerper hier een aardige kandidaat voor blijft. Hoewel het er wel op lijkt dat best veel mensen hier vrij concrete –eigen– ideeën over hebben. Dus misschien wordt het wel als relevante vraag ervaren, maar niet zozeer als probleem. In ieder geval niet door degenen met hun eigen pet theory.

Iets dat in ieder geval voor studenten een relevante vraag lijkt –een zeer zeker als probleem ervaren wordt!– is hoe ontwerpvoorstellen beoordeeld worden. Wat voor soort kritiek kun je verwachten? En hoe verdedig je je voorstel?

Vandaag bij de presentaties waar studenten met ontwerpvoorstellen kwamen, kwamen er een paar mooie langs.

“Ik denk niet dat het mechanisme gaat werken zoals jullie denken dat het werkt,” gebaseerd op natuurkundige en mechanische kennis en redenatie.

“OK, ik geloof dat het zou doen wat jullie denken, maar treedt er daarnaast niet ook dit effect op, dat ongewenst is?,” op basis van simulerend denken.

“Wat is nu eigenlijk het waardevoorstel? Ja, het doet wat jullie denken, maar oplossing X zou toch met minder kosten een veel beter resultaat opleveren?!”, op basis van kennis van alternatieven en kritisch denken over de probleem- en doelstelling.

“Oe, gaaf, maar waarom hebben jullie onderdeel Y niet op manier Z gemaakt, dan wordt het effectiever/goedkoper/sterker/…,” op basis van creatief mee-ontwerpen en kennis van alternatieve deeloplossingen.

Dit zijn:

kritiek dat het gedrag incorrect voorspeld is
kritiek dat het gedrag onvolledig voorspeld is
kritiek op de waarde van het ontwerp
kritiek dat het ontwerp niet optimaal is

Wat ook leidt tot kritiek…

… dat het ontwerp onvolledig is
… dat het ontwerp incorrect (onmogelijk is)

Er kan kritiek zijn op minstens 3 niveau’s:

op de probleemstelling (niet duidelijk/volledig/concreet/specifiek)
op het concept (niet de beste aanpak, inherente nadelen/conflicten)
op de uitvoering (niet volledig, optimaal, goed voorspeld)

Of anders gezegd, op 3 aspecten:

functie
concept
optimalisatie

Do Less, Get More

We ask too much of our students. I believe that by asking less, we would get more. In the large introductory course that I teach in, at least, students are asked to do so many different new things in such a short amount of time that they don’t get a fair chance to really master much of it.

And everybody knows something isn’t right, judging by the amount of complaints from colleagues about all the things students don’t know, understand, or can’t do later on in their Bachelor’s. And whatever it is that’s wrong, it must be our fault, I believe. Because even if students were lazy –which they’re very much not– it’s us who passed them in all those courses.

The metaphor of “setting a high bar” sounds good, but isn’t right. Most courses consist of many different bars to jump over and hoops to jump through. And we give points for reaching almost high enough, but not quite. This means that by cramming our courses full with as much as we can get away with is guaranteed to lead to students passing while having mastered precious little of all that material.

So why do we do it? I see at least three reasons.

Who’s Wasting Whose Time?

Some teachers seem to feel that students who don’t pay attention in class, or that don’t show up, are wasting our time.

But it’s we who are spending their time. And if they don’t show up, or do something else, they apparently feel they can get more value out of that time than they see us to be offering.

Now, they might be wrong about this, but odds are that at least they’re acting rationally in according with their current judgement. If they don’t see that they’re wrong, you can’t blame them for acting against their own interests.

Production Is Required for Learning Design

Teaching design seems to require a focus on products. Not end-products, but the products of the process of design.

Students that are just starting out as designers do not –cannot– see what experts see. They do not see the complexity and lurking problems and hidden opportunities in their ideas. This must be brought in the open somehow, so that the student can be confronted with these unexpected features of their ideas.

A Note on In-Lecture Discussions

I gave a lecture on different ways of modelling where I asked the students to discuss with neighbors which role each of 6 ways of modelling might play in the design process, what they show, and what they hide. The goal was to explain that each does in fact highlight and leave out different aspects, so you should use as many of them as possible, and keep using different modelling methods throughout the project.

But they were at a point in their design projects where it was hard for them to understand the relevance of this, I think, so weren’t curious to know.

But also, for these in-lecture discussions to work, perhaps there needs to be a right answer. It helps when they can try to get it right, and then find out whether they did.

So not “discuss what a good problem statement looks like”, but “each of these three problem statements is good in one way, and bad in another way’. Find what is good and bad about each statement”.

Two Views of the Teacher-Student Relationship

One follows from the view that teachers know something students do not and that students do not know what’s good for them. Both true. But people seem to draw as a conclusion from this that teachers need to tell students what to do and that students should simpy listen. This does not work.

Another view sees students as rational adults, who can and should decide for themselves what they want and how to achieve this. This also seems to me a solid assumption. But proponents of this view draw from it the conclusion that we should let students take the lead, that they should decide how to approach their projects and what learning activities to engage in. This does not work, I believe, because it conflicts with the above truth that students –in the subject of what’s being taught– do not know what’s good for them. Teachers do. Or should, in any case.

But students need some understanding of how what they’re being asked to do is useful or necessary.

Teachers must understand how naïve and mistaken models can be dislodged and developed into the ones the teacher wants to teach. This is diffucult. But one way this most surely cannot be done is to simply tell the student and expect them to take your word for it.

On Design as Research

Designing a building or product forces you to solve a range of problems, to answer a set of questions. A car needs an engine cover, doors, a trunk that opens, openings in the body for headlights, etcetera. A building needs a stable structure, doors, windows, insolation, waterproofing, perhaps floor levels, it should provide functional spaces, etcetera. There are issues to deal with at the level of the whole design, and there are parts, fragments, and details to work out.

Dealing with such a set of issues, and their interactions, conflicts, and overlap, leads to a thorough interrogation of the material or technology you’re working with. Some of the answers will be specific to this one design. But a few of them will be of more general value. They could become a standard component, technique, or pattern. A standardized detail, combination of techniques, or construction method, for instance.

Such experiments can test and/or explore. They can ask, does it work? Or they can ask, what if?