Professor KIM'S COURSE OFFERING IN FALL 2016

 

If you’ve taken Statistics before, yet not sure how to properly apply it to solve your own problem.

Learn to apply statistical methods to planning, data collection and analysis through the experiment of your own design.

SYS 4581-004 / SYS 6581-004 Selected Topics in Sys Engineering: “Design of Experiment”

• This course is open to the 3^rd– & 4^th-year undergraduate and graduate students in SEAS.

• This course counts as a Human-Factors application elective for SIE students.

Instructor

Inki Kim

Office: 101H Olsson Hall

Office hours: To be scheduled through Collab

E-mail: inki@virginia.edu

Course Logistics

Class: T/TR 15:30-16:45 CHEM 005

Prerequisite: APMA3120 or equivalent (e.g. STAT2120, 3120)

Class Notes: To be shared on <Collab>

Textbooks (optional):

• “Design and Analysis of Experiments”, 8^th Edition, by D. C. Montgomery, 2015, John Wiley.
• “Experiments- Planning, Analysis and Optimization”, 2^nd Edition, by Jeff Wu and Mike Hamada, 2009, John Wiley.
• Online tutorials on core statistical concepts: Search “Intuitive Statistics” (by Reid Bailey) on YouTube

Why should you care about Design of Experiment?

The successful inventor and entrepreneur, J. Dyson once claimed that “You have to change one thing at a time. If you make several changes simultaneously, how do you know which has improved the object and which hasn’t?” If you agree with him, probably you should consider taking this course to get a better perspective[1] of statistical thinking.

Research in engineering and science requires to formulate important hypotheses that can be tested, to select the most appropriate study design, and to conduct experiments with sufficient transparency so that the results ensure reproducibility[2]. Since the early twentieth century, DOE has laid theoretical and practical foundations for effective experiment, variable screening, design and improvement in medicine, agriculture, and manufacturing. Its contemporary applications extend to business, marketing and service areas to maximize productivity.

Especially, system engineers are challenged to analyze complicated systems with multiple inputs whose simultaneous variations impact the output. DOE helps identify specific input variables or their functions that significantly change the output, based on the economical size of empirical data. DOE also offers useful techniques to control nuisance effects when collecting data from physical systems which may otherwise have inflicted systematic bias on the data.

Academic researchers commonly struggle to confirm their study design is sound lest red flags should dilute the credibility of their findings. Misuse of design and analytic methods can cause your work rejected irreversibly or refuted for obvious reasons[3]. Even a huge body of published works are criticized for Questionable Research Practice due to inappropriate study design, which result in the lack of repeatability and reproducibility[4]

[1] Spall, October 2010, Factorial Design for Efficient Experimentation, IEEE Control Systems Magazine

[2] Shavelson & Towne, 2002, Scientific research in education. Washington, DC: National Research Council, National Academy Press

[3] Campbell, S. K. 2004. Flaws and fallacies in statistical thinking. Courier Corporation.

[4] Norvig, P. 2007. Warning signs in experimental design and interpretation.

What you should learn from this course

Through this course, you can select an appropriate design method suitable for answering your own research question, and justify your selection using various tests. You will learn it through lecture (70%) and project (30%). For the project, you will collaborate in a group to practice design, collection and analysis of experimental data. Despite unknown or uncontrollable variabilities in data, you should determine study design well ahead of data collection. This decision is particularly challenging when the input-output structure is unknown, or even underlying assumptions are compromised. This course intends to prepare you to handle such tricky situations.

In particular, you will be able to do the followings at the end of the course.

• Explain the logic of hypothesis testing, Analysis of Variance and Covariance, Regression, and Response Surface Method
• Evaluate various design methods (factorial & fractional-factorial designs) in terms of their advantages and disadvantages
• Determine proper techniques to control variance of data
• Determine acceptable number of subjects with respect to Type-I and Type-II errors
• Formulate empirical observation and error by using stochastic model
• Identify significant inputs and input-levels from a unknown system
• Describe advanced methods for robust design
• Report and interpret statistical outcomes by using Excel, SPSS, SAS and R

Evaluation Components and keys to succeed in this course

Lecture (70%)

The lecture materials cover a wide range of statistical concepts and techniques, and the lecture is organized into short cycles of teach one, apply one, and test one. Your performance of comprehending lecture is evaluated by practice (30%) and knowledge test (40%). For practice, you are assigned specific tasks to complete within 1-3 days by using a statistical software package.

For high performance in practice, it is important that you know what exactly the software package does for a particular function and associate it with the lecture contents. When in doubt, you are encouraged to refer to reliable sources like software manual or articles and cite them, rather than asking your peers for self-assurance.

Knowledge testing is conducted in the form of quizzes or exams. The best practice is to keep yourself up-to-date with the lecture. If you lag behind the lecture schedule over a week, please don’t hesitate to consult the instructor for help.

Project (30%)

You work in a group to plan, conduct and analyze miniature experiment that involves human subjects. Each group will be provided with Wireless Sensor Network to track body movements, eye gazes, brain activities or muscle activities. Research hypothesis and target task will be determined in discussion with the instructor. Your group performance will be evaluated by effective variance control, flawless study design, thorough analysis and complete justification. It is beneficial to team up with people who could complement your own skill set. For successful results, spend ample time to discuss the progress with your team-members and the instructor.

Discussions (Extra-credit)

You are always welcome to bring up topics related to the lecture material or project to discuss them in class. By preparing and presenting short slides, you should introduce the topic and lead a brief discussion in class. The topics include supplement to or disapproval of methods in the lecture, design of your own research, or even better, proposal of novel method. Ahead of preparation, consult your plan with the instructor.

Grading Policy

TOTAL SCORE BREAKDOWN
Quizzes	20 %	[=5% * 4]
Exams	20%	[=10% * 2]
Practice	30%	[=10% * 3]
Project	30%

Course Schedule (tentative, in SP2016)