English

Designing Effective Multimedia for Physics Education

Muller, D. A. (2008). Designing effective multimedia for physics education. Sydney: University of Sydney.

https://www.sydney.edu.au/science/physics/pdfs/research/super/PhD(Muller).pdf

–

Derek Muller (creator of ‘Veritasium’ on YouTube), in his PhD thesis, shows that science education videos need to start with students’ misconceptions to be effective. A straightforward exposition can be worse than no instruction at all, because students do not change their mistaken views but do become more confident that they know how something works.

Effective Educational Videos: Principles and Guidelines for Maximizing Student Learning from Video Content

Brame, C. J. (2016). Effective educational videos: Principles and guidelines for maximizing student learning from video content. CBE—Life Sciences Education, 15(4), es6.

https://doi.org/10.1187/cbe.16-03-0125

–

A clear, short, and highly readable review paper that discusses research on educational video and provides practical advice on optimizing three aspects of using video in education effectively:

Managing cognitive load:
- Use signaling to highlight important information.
- Use segmenting to chunk information.
- Use weeding to eliminate extraneous information.
- Match modality by using auditory and visual channels to convey complementary information.
Maximizing student engagement:
- Keep each video brief.
- Use conversational language.
- Speak relatively quickly and with enthusiasm.
- Create and/or package videos to emphasize relevance to the course in which they are used.
Promoting active learning:
- Package video with interactive questions.
- Use interactive features that give students control.
- Use guiding questions.
- Make video part of a larger homework assignment.

Or, more concisely (quote from the conclusion):

Keep videos brief and targeted on learning goals.

Use audio and visual elements to convey appropriate parts of an explanation; consider how to make these elements complementary rather than redundant.

Use signaling to highlight important ideas or concepts.

Use a conversational, enthusiastic style to enhance engagement.

Embed videos in a context of active learning by using guiding questions, interactive elements, or associated homework assignments.

Understanding in-video dropouts and interaction peaks in online lecture videos

Kim, J., Guo, P. J., Seaton, D. T., Mitros, P., Gajos, K. Z., & Miller, R. C. (2014, March). Understanding in-video dropouts and interaction peaks inonline lecture videos. In Proceedings of the first ACM conference on Learning@ scale conference (pp. 31-40).

https://doi.org/10.1145/2556325.2566237

–

Empirical study of drop-out rates and viewing statistics in a large number of edX course videos.

Peaks in viewing numbers – indicating rewinding and re-watching – occur around visual transitions. Students go back to a slide that’s suddenly gone and use the transitions as visual ‘bookmarks’ to re-watch or rewind to a particular explanation or section of the video.

Therefore, avoid taking away diagrams and other visual aides too soon and/or abruptly. Include clear visual anchors (e.g. title cards) at the start of sections, provide timestamps, or cut up longer videos into shorter ones.

Recording Quick Feedback Videos

As an alternative to written feedback, I make simple videos. Once you have it set up, this actually takes less time than responding with text. It’s more fun, and research suggests it’s also more effective:

Students found screencast technologies to be helpful to their learning and their interpretation of positive affect from their teachers by facilitating personal connections, creating transparency about the teacher’s evaluative process and identity, revealing the teacher’s feelings, providing visual affirmation, and establishing a conversational tone.

Cost-Benefit Considerations in Design Arguments

In architecture, it may be perfectly acceptable to present predictions that are based purely on theoretical ideas about how people will behave and feel in response to a proposed building. Human behavior is so complex, and buildings so large, that such claims can be utterly impractical to test or otherwise validate. We have little choice but to trust the architect’s expertise, or accept an argument by analogy.

An engineer presenting a design for a novel surgical device, however, is expected to present a prototype that has been tested on simulated or even actual tissues, in addition to a theoretical model that predicts and explains its behavior. The physics of metal devices have been reliably modeled, and it is perfectly feasible to produce one-off prototypes and set up empirical experiments to validate these predictions with a reasonable investment of resources.

In design disicplines, we expect or do not expect certain types of evidence based on the possiblity and cost of supplying them. Engineering arguments are subject to cost/benefit considerations, similar to the designs themselves that the arguments are about.

Diminishing Validity of Concept Selection as an Argument down the Line

The detailed development, implementation, and operation of a design usually represents a significant investment. This makes it a good idea to first explore a number of possible approaches before committing to a single concept.

But concept selection is a strategic choice. The decision comes down to a judgement of which concept looks most promising, not to a determination of which one is certain to have the best possible performance. And at the end of a completed design project, you can never be certain that a choice to go with a different concept would not, in fact, have led to a better outcome. It is just that at the time, this concept looked best, and that therefore it was the one selected for further investment of development resources. Who knows what would have happened if those same resources had been invested into a different concept?

Soft Spots in Design Arguments

A design is always presented as a means to achieve a goal of some kind, in a certain situation or context. To argue that the proposed design will actually do this requires a bit of a detour, however. First of all, goals are usually complex, ambiguous, and ill-defined. They need to be made operational in a set of objectively testable criteria (functional requirements, performance criteria, and constraints). Secondly, it is not obvious from the plans for an artefact how that thing will do its work, precisely. Its behavior needs to be predicted. Predicted behavior can be evaluated in terms of the operational criteria. This is the claim that designers can actually establish. It serves as a proxy for the actual motivation behind the design, the expectation that the design will actually achieve its goals in the real world.

The translation of a complex goal into an unambiguous, operational set of criteria is not straightforward. Different people can legitimately interpret the same goal differently. The argument for a design proposal needs to establish, therefore, that this translation is a good one. Does it capture all the relevant aspects? Is anything lost in the definitions and quantifications employed? Is it possible to formally meet these criteria, while clearly failing to achieve the actual goal?

Predicting the behavior and performance of the proposed system can look like the straightforward, rational, objective part of a design project. But this is not straightforward either. To predict something’s behavior, we need to model it. Models are always simplified, partial and idealized representations. Abstract models can be validated through controlled tests with a prototype, but tests also only pick out parts of the actual operation of a system, and prototypes are, like abstract models, partial, idealized representations. In fact, they often introduce properties that the actually proposed design would not have. Here as well, the argument relies heavily on judgements of definition, translation, and interpretation.

Discovery and Justification in Design Proposals

What is the logic of design proposals? What argument is or needs to be made when you present a design? What is it that a design proposal does and what criteria must it meet to perform this function?

Engineering can be contrasted with science in that it is not only descriptive, but also prescriptive. The goal of a scientific paper is to describe and explain the world as it is. An engineer’s design prescribes or at least proposes what should be done or changed in the world: ‘if you have a certain goal, then here is a plan to achieve it’.

This makes a design proposal, in rhetorical terms, an argument about policy. Much of it may be concerned with facts and causation, in the end it is a question of means, ends, and value. Such an argument is always relative. The proposal can be compared to existing options, alternative proposals, and to leaving the situation unchanged. And while scientific claims aim at universality, designs are always context-dependent, appropriate to a specific time and place.

If this is the argument we need to make, how do we argue it?

What About the Logic of Design Proposals?

Next to a description of an artefact, plans for its production, and plans for its use, the product of a design project must always be a design proposal. There is the rare case where a design “speaks for itself”, but even in that instance, what that design says amounts to an argument that proposes the design’s actualization. And in arguing for a boss, client, or teacher, to make it like this, automatically means to not make it like that, or to leave the world as it is and keep making the same thing as before, or to make nothing new at all.

In practice, the goal and measure of success of such a proposal is that it persuades. In academic circles, we should instead be interested in whether the argument is any good in terms of its logic and evidentiary weight. Also in practice, however, those on the receiving end of a design proposal will want to judge how successfully the arguments offered actually justify a belief in the value of the design under consideration, and to poke through any rhetorical flourishes and salestalk that may be involved. In fact, I would argue that engineers –as opposed to those with sales and business titles– are under a moral obligation to strive for the same: an honest presentation of the merits of a design, accurate rather than merely giving the impression of accuracy. If “trust me, I’m an engineer” is to remain a valid request, we should strive to be trustworthy.

What is the logic of design proposals? What, exactly, are the claims that are made when designers present the results of their efforts? And how are and can these be justified?

Is the result of design always a proposal? Do designs published in academic journals fit this description?

At first glance, they don’t. Their message is more “Here is what we made. It’s really good/interesting/valuable/impressive.” But isn’t this the same as saying “This is how we should make these kinds of things for these kinds of situations.”? Or, “This is how we should solve this problem, or reach this goal.”?

Law and Order in the Studio, Part II

If a design presentation and critique is compared to a courtroom setting, then what roles do the design, designer, and design judges play?

Perhaps the design critique is a reverse trial. The design is ‘on trial’, but as something to be accepted rather than judged. The designer is the prosecutor, with the burden of evidence to establish beyond a reasonable doubt that the design has merit, or even that it is the best possible answer to th design brief. The critic (boss, client, teacher) plays to roles of defense council, judge, and jury. While the designer tries to make their case, they look for holes, weaknesses, and alternative explanations. When the case of the prosecutor (the designer) stands up, the design is judged as ‘good’, or ‘acceptable’.

It might be fun to separate these three functions in an educational setting. The teacher can be judge (only making sure everybody follows the rules and plays fair), an external critic can be the defense lawyer (doing their best to point out weaknesses and poke holes in the designer’s work), and other students can be the jury (rendering their verdict after hearing all the evidence).