MATH5510 Topics in Mathematics for Teachers Summer 2016
From MathWiki
Add: ignoring the baseline. Gelman and Nolan p. 149
Annotated Links  Workshop 
Example of not taking independence into account, i.e. not using a hierarchical model (http://bmcbiol.biomedcentral.com/articles/10.1186/s1291501602278?utm_campaign=BMC30439C&utm_medium=BMCemail&utm_source=Teradata)
Calendar
Day  Plans and Links 

1 May 10 
Introduction: Fun (cool?) things to think about in Stats. line 2 
2 May 12  
3. May 17  
4. May 19  
5. May 24  
6. May 26  
7. May 31  
8. June 2  
9. June 7  
10. June 9  
11. June 14  
12. June 16 
Course Outline
 General ideas: Justintime background: e.g. weighted means before Agresti Diagrams
Does X 'cause' Y?: Causality and Statistics
 The importance of asking how?, not just what? Many statistics courses condition us to operate on and transform information. We easily forget the critical importance of askeing HOW the information was generated. The whole approach to information should be completely different depending on the answer to HOW. We explore the details. All statisticians can do with data is a set of complex rituals that is devoid of meaning unless we first understand how the data were generated.
 Agresti (find better name) diagrams: with data and speculative (find a better word maybe)
 Gelman (xxxx) "When Confounding Variables are Out of Control" (http://blogs.discovermagazine.com/neuroskeptic/2016/04/02/confoundingvariables/#.Vx5ZHfkrIUF)
 Westfall J and Yarkoni (2016) Statistically Controlling for Confounding Constructs Is Harder than You Think. (http://www.ncbi.nlm.nih.gov/pubmed/27031707)
 Gelman (2016) on Confounding (http://andrewgelman.com/2016/02/22/itstoohardtopublishcriticismsandobtaindataforreplication/)
Visualizing Correlation
 Kahneman's pilots: link with causality: How two seeming contradictory views can be right
Probability, Bayes and pvalues
 Basic probability (avoid random variables  delay until probability)
 Bayes: Trees and 2x2
 Monte Hall and the importance of modelling information (again: asking HOW, not just WHAT).
 Finite models (finite parameter space and finite sample space) (assumes some knowledge of matrices and linear algebra)
 Gigerenzer: did UK doctors use Bayesian nomograms. Does your doctor use a Bayesian nomogram. Building a nomogram app.
 Crisis of reproducibility: asking How: The effects of selection. Funnel graphs?
Visualizing Multiple regression
If time permits which seems unlikely.
Statistics as a Profession
Do We Disteach Statistics?
 Delmas, R., Garfield, J., Ooms, A. and Chance, B. (2006) "Assessing Students' Conceptual Understanding After a First Course in Statistics", Paper presented at the Annual Meetings of The American Educational Research Association. (https://apps3.cehd.umn.edu/artist/articles/AERA_2006_CAOS.pdf)
 Some items showed "an increase in a misconception or misunderstanding from pretest to posttest."
 Random assignment is confused with random sampling or thinks that random assignment reduces sampling error
 n = 483 precourse error = 36.4 postcourse error = 49.7
 Causation can be inferred from correlation.
 N = 470 precourse error = 26.8 postcourse error = 36.4
 Grand totals are used to calculate conditional probabilities 456 27.2 37.3 .001
 N = 456 precourse error = 27.2 postcourse error = 37.3
 Rejecting the null hypothesis means that the null hypothesis is definitely false.
 N = 456 precourse error = 25.7 postcourse error = 35.1
 Random assignment is confused with random sampling or thinks that random assignment reduces sampling error
 Some items showed "an increase in a misconception or misunderstanding from pretest to posttest."
R
 Installing R and RStudio
 Basic R, graphics, some lattice
 Rmarkdown,
 Latex
 Some data structures?
 p3d
 Gapminder
 Shiny
 R Tricks for Kids  Improve and add new ideas
 Teaching Statistics: a bag of tricks: Chapter 8 ff
 http://www.rossmanchance.com/applets
 http://www.socr.ucla.edu/
 Interacting with plots
 Click, double click, hover and selection box (http://shiny.rstudio.com/articles/plotinteraction.html)
 Identifying: using 'brushedPoints' (http://shiny.rstudio.com/articles/selectingrowsofdata.html)
 Advanced interaction (http://shiny.rstudio.com/articles/plotinteractionadvanced.html)
Course Work
There are X components:
1. Short daily quiz [20%]
The quiz is assigned at the end of each lecture and is due by 11:59 pm of the next day. It should be submitted through Moodle.
2. Project and presentation [30%]
In teams of 2 assigned semirandomly. Prepare an essay and a lesson suitable for students in senior high school or in an introductory university statistics course on one of the following topics:
 Bicycle helmets: What does the evidence say about their effectiveness?
 The Gardasil Scare: Last year the Toronto Star published an article claiming that Gardasil,
a vaccine against HPV, had disastrous side effects. What does the evidence say?
 If you enjoy programming, you can improve a collection of interactive 'shiny' apps by
rewriting them to include graphical interaction.
 ... MORE
3. Something else ????
????
4. Exam [30%]
1. Wiki contributions [10%]
Contribute to a course blog at least once a week. Your contribution can take many forms: a link to an interesting article or piece of news together with a comment, a question about some aspect of the course, an answer to someone else's question, a link to information relevant to the course, etc. If you are an outlier in the quality of your contributions, there's an opportunity for a few bonus marks here.
2. Assignments [25%]
Four assignments on material covered in the course. Some of the assignments are individual assignments and some are team assignments.
Assignment  Due  Description  Team or Individual 

Description of Assignment 1  Tuesday Jan 5 and Thursday Jan 7 11:59pm  Install software (R, RStudio, Git) and get a free account on Github.com. Complete [[[Template:Math4939survey]] this course survey]. Bring your laptop, well charged, to class on Wednesday, January 6 and Friday, January 8.  Individual 
Description of Assignment 2  Sunday January 11:59pm  Analyze and interpret Arrests data set in the effects package in R  Teams using R Markdown and Github 
Assignment 3  Thursday February 11 11:59pm  Prepare and post on the wiki discussions of selected questions. Teams posting on the course wiki
 
Assignment 4  Thursday March 3 11:59pm  Comment on discussion on selected questions. Teams posting on the course wiki

3. Odd jobs [5%]
There's always a host of interesting small questions and problems that come up in a course like this one. You get grades for performing an 'odd job' when you take on the responsibility to research and answer one of these questions and post the results on the wiki, perhaps in the form of a short tutorial helping others solve similar problems in their work. Insert a link to your work in the Odd Jobs page.
4. Team project due March 21, 2016 [30%]
Change in deadline: Sunday, March 27, 2016
You will work on a team project in which you solve a real problem involving real data and prepare a report including analyses, graphical displays and a careful interpretation of your findings. The report has three parts:
 A '.R' script using rmarkdown that produces a detailed analysis and presentation of your work, including diagnostics, etc. This output can be quite long.
 A '.R' script using rmarkdown that produces an attractive and readable report with your main findings. You need to include all relevant references, data sources, etc. Aim for a maximum of 30 pages.
 Slides for a 10minute presentation discussed below. The slides can also be prepared with Rmarkdown using the ioslides format (http://rmarkdown.rstudio.com/ioslides_presentation_format.html).
You will collaborate using R, R Studio, R Markdown, git and github. The grade is based on the overall quality of the project (10%) and on your personal contribution to it (10%) and on your understanding of the issues and concepts in the project as shown in the final presentation and in project meetings with instructor. (10%).
You will prepare a brief summary of your project for a 10minute presentation on Wednesday, March 23, and Friday, March 25. The 10minute limit is strict. Be mindful that it takes careful preparation and rehearsing to give a good presentation in such a short time. You must rehearse as a group ahead of time. The presentation will be followed by a 5minute question and discussion period.
Here are the four projects you can choose from. They are current and recent case studies used in the Statistical Society of Canada's annual case study competition. I choose these because the data for them is freely available to you although it might take some initiative to obtain it.
Projects: Recent  and one current  case study from the SSC Case Study competition:
 2016 Case study I (http://ssc.ca/en/meetings/2016/casestudies#CaseStudy1)  Pearson
 2015 Case study I (http://ssc.ca/en/meetings/2015/casestudies#case1)  Fisher
 2014 Case study I (http://www.ssc.ca/en/meetings/2014/casestudies#case1)  Neyman
 2015 Case study II (http://ssc.ca/en/meetings/2015/casestudies#case2)  Bayes
Each team should choose a different one. I hope that you will agree on which one each team chooses. If not, we will have a draw at the class on Monday, February 8.
I am not sure if you forget to post a link for the projects or if there is a place where we can find them. [AA] Sorry, I must have made an error in cutting and pasting. It's corrected now. [GM]
5. Individual or selfselected team project [10%]
This project provides you with a chance to pursue your own interest. If you plan to apply to more technical jobs, you might want to do an R package on Github, a good addition to your Github portfolio that technical employers and startups look at to assess job applicants. If you plan to apply to jobs at large institutions such as banks, you can work on SAS by, for example, implementing your project in SAS as far as possible. Submit a project proposal to the instructor by March 1. The project is due on April 4, 2016.
6. A final 2hour exam [20%]
The exam will be held in the regular exam period. A major component will consist of questions probing your understanding of statistical concepts reflected in the list of 20 22 questions.
Possible Topics
Syllabus
What is Statistics?
The precise boundaries of a discipline should never be made rigid. In fact, our organization of knowledge into separate disciplines is a curious thing in itself. It seems expedient to link some groups of ideas together because awareness of them seems to promote development in the investigation of questions that arise from these ideas.
 Inference under uncertainty?
 Extracting meaning from data?
Latest ideas
 Use R course for 1st six halves
 Incorporate R Markdown (http://rmarkdown.rstudio.com/), Shiny examples (http://shiny.rstudio.com/) and Tufte style (http://rmarkdown.rstudio.com/tufte_handout_format.html)
 Invite Heather
 Include regression ellipses, regression to the mean and Kahneman's pilots
 How to include conditioning?? Multiple regression? Stratification?? See how it's done.
 Multiple regression?? and ellipses??
 A chance to play and learn neat stuff:
 Some basic probability and statistics culminating with likelihood and Bayes with prior and posterior odds and likelihood ratio.
 Together with some R using Markdown
 An assignment every day to hand it the next?
 Introductory: Why! Something about current statistics, the emerging crises and where's the resolution. Our current (conflicting) approaches are adopted by various groups as if they were absolutely correct recipes. Sometimes they work but often they don't but most don't see the problem. Two fundamental problems:
 Interpretations of probability in application to the interpretation of scientific evidence.
 The nature of causality and its inference from empirical observation. (mention briefly as motivation but don't belabor until ready later)
 Broad issues:
 Bayes
 Inversion of conditionality
 Interpreting health tests
 pvalues and Sally Clark
 Monte Hall: Using the model that generated information
 Overview of issues: maybe history of probability and statistics: Bayesian talk  but need probability review first  background for Gigerenzer et al. Helping Doctors and Patients make Sense of Health Statistics
 Causality:
 The critical role of randomization
 The usual impossibility of randomization and the limitations of experimentation (e.g. assessing effect on rare but important effects. Use Vioxx as an example)
 Probability for Patients
 Causality
 Probability  cognitive errors
 Bayes
 Read: Gerd Gigenrenzer et al. (2008) "Helping Doctors and Patients Make Sense of Health Statistics", Psychological Science in the Public Interest (http://pubman.mpdl.mpg.de/pubman/item/escidoc:2100705/component/escidoc:2100704/GG_Helping_2008.pdf)
Topics
 Import http://capstone.stats.yorku.ca/index.php/2016/Statistics#The_Crisis_of_Reproducibility_and_the_ASA
 Causality: what do statistical analyses mean?
 Include implicit conditioning on colliders as creating 'spurious' correlation, e.g. among people admitted to hospital. Emphasize possibility of implicit conditioning, e.g. implicit selection.
 Climate change
 Racial profiling
 Drug side effects
 Bayesian vs frequentist: why and when it matters?
 Sally Clark and Lucia de Berk
 Better Bayes: focus on prior odds, posterior odds and Likelihood ratio. Or on relative prob:
posterior relative prob = prior relative prob. x likelihood.
 Emphasizing Bayes Formula would be like teaching the formula for the probability of the union in terms of odds.
 Regression to the mean: Kahneman's pilots: predictive vs causal
 Reproducibility: a crisis in statistics, or a crisis in science?
In parallel?:
 Gelman's Teaching Statistics: A Bag of Tricks
 Programming: Access to Gapminder
+ a little bit of real programming
 Validity of statistics  Vincent Granville (http://www.datasciencecentral.com/profiles/blogs/pvaluesthegoldstandardofstatisticalvalidityarenotas?utm_content=buffer90c43&utm_medium=social&utm_source=linkedin.com&utm_campaign=buffer)
 Regina Ruzzo (2014) Scientific Method: Statistical Errors in Nature (http://www.nature.com/news/scientificmethodstatisticalerrors1.14700)
Over the same period, but especially since the 1990s, there has been an increasing disconnect between the traditional FisherNeymanPearson (FNP) math statistics course and the demands for complex analysis in many application areas. The failure of classical maximum likelihood methods to deal effectively with complex models and the success of MCMCbased methods has led to a similar situation: The undergraduate FNP course does not prepare students for these models, and Bayesian MCMC retraining courses are needed to prepare graduates for these applications.  Murray Aitkin in Amstat News, March 2014, p. 28 (http://magazine.amstat.org/wpcontent/uploads/2014/03/March2014.pdf)
 Andrew Gelman insults Data Science!! (http://www.datasciencecentral.com/profiles/blogs/statisticianshaveabiasedviewondatascience)
MATH 5510: Topics in Mathematics for Teachers
Tentative Topic:
A challenge Statistics was late to face: The role of Prediction and Causation in understanding the world.
Understanding Bayes vs Frequentist inference: see https://youtu.be/BcvLAwJRss from Walter Whiteley
Fridell (2004) "By the Numbers" (http://www.policeforum.org/assets/docs/Free_Online_Documents/RaciallyBiased_Policing/by%20the%20numbers%20%20a%20guide%20for%20analyzing%20race%20data%20from%20vehicle%20stops%202004.pdf) on statistical analyses of racial profiling.
The Crisis of Reproducibility
Why so many statistically significant results fail to replicate. Is something fundamentally wrong with 'statistical significance'? Or is it more how we use it or interpret it that is the problem? There a psychology journal that is banning hypothesis testing entirely (http://www.nature.com/news/psychologyjournalbanspvalues1.17001).
 In Economics (http://www.sciencemag.org/news/2016/03/about40economicsexperimentsfailreplicationsurvey?utm_campaign=emailnewsweekly&et_rid=35385820&et_cid=319219)
The American Statistical Association has jumped into the act by issuing a report on pvalues that will have, I believe, seismic effects in the statistical profession and in research in general.
 Ronald L. Wasserstein & Nicole A. Lazar (2016): The ASA's statement on pvalues: context, process, and purpose, The American Statistician, DOI:10.1080/00031305.2016.1154108 (http://dx.doi.org/10.1080/00031305.2016.1154108)
To link to this article: http://dx.doi.org/10.1080/00031305.2016.1154108
 Early March 2016: press releases and public statements by the ASA on pvalues:
 538.com: It's time to stop misusing pvalues (http://fivethirtyeight.com/features/statisticiansfoundonethingtheycanagreeonitstimetostopmisusingpvalues/)
 Nature: Statisticians Issue Warning Over Misuse of pvalues (http://www.nature.com/news/statisticiansissuewarningovermisuseofpvalues1.19503)
 Amstat News (http://amstat.tandfonline.com/doi/abs/10.1080/00031305.2016.1154108#.Vt2ee5MrI0p)
 Interview today on Retraction Watch about the ASA statement with the ASA president (http://retractionwatch.com/2016/03/07/wereusingacommonstatisticaltestallwrongstatisticianswanttofixthat/)
 The Cult of Statistical Significance
 Stephen T. Ziliak and Deirdre N. McCloskey (2009) "The Cult of Statistical Significance", JSM 2009  Section on Statistical Education (http://www.deirdremccloskey.com/docs/jsm.pdf)
 Focuses on the "significance" is not "importance" problem. Scientists need to pay attention to effect size as well as evidence. Interesting reference to the Vioxx case. Clinical trial in 2000: Five patients had heart attacks in the Vioxx arm compared with 1 in the naproxen group (e.g. Aleve). Not significant so Merck claimed no difference in the effects of the two pills on heart attacks. Fallacy 1: Concluding the Null, i.e. Absence of evidence is not evidence of absence. Fallacy 2: Ignoring importance in the absence of evidence.
 Stephen T. Ziliak and Deirdre N. McCloskey (2009) "The Cult of Statistical Significance", JSM 2009  Section on Statistical Education (http://www.deirdremccloskey.com/docs/jsm.pdf)
Causality: A scientific problem or a statistical problem
 Have a look at work by Tina Grotzer.
Taxonomy of Fallacies
 Base rate fallacies (https://en.wikipedia.org/wiki/Base_rate_fallacy): ignoring the prior when you shouldn't
 Use probability of evidence given hypothesis as a measure of strength of evidence (e.g. pvalue, sensitivity, specificity) and ignoring the base rate, e.g. the prior probability of the hypothesis. Some notable examples:
 Using observed likelihood instead of a posterior to form a judgment: You see an unkempt person with dirty glasses walking on campus? It is more likely to be a grad student in mathematics or a grad student in business. The likelihood (in the sense of P(datahypothesis) is higher for mathematics than it is for business. But if you take the base rate (the number of grad students in either discipline) into account the posterior probability favours business.
 Representativeness heuristic (https://en.wikipedia.org/wiki/Representativeness_heuristic) and representation bias: one of the cognitive biases identified by Kahneman and Tversky. A mode of thought used by 'System 1'. It's easier to visualize P(DataHypothesis) and, having assessed it for various hypotheses, one is inclined to use that and not follow with the much more complex visualization that considers the base rate (prior) to compute a posterior. The relative probabilities for the posterior are very easily calculated by taking the product of the prior with the likelihood. Absolute probabilities are harder because they require norming. The progress on solving the norming problem, e.g. with MCMC, is one of the major recent advances that make Bayesian analysis much more computationally feasible than it used to be.
 Prosecutors's fallacy (https://en.wikipedia.org/wiki/Prosecutor%27s_fallacy): Using the equivalent of a pvalue as a measure of evidence in a trial, ignoring the prior. See R v. Sally Clark  Convicted on Statistics? (http://understandinguncertainty.org/node/545). Consider the parallel case of Susan Nelles (https://en.wikipedia.org/wiki/Toronto_hospital_baby_deaths) in Toronto.
 Misinterpreting the probability of disease from medical tests
 See Gigerenzer. The correctness paradox. If a test for disease D has specificity = sensitivity = 95%, this means that Pr(CorrectD) = Pr(Correct not D) = 0.95. So, necessarily Pr(Correct) = 0.95 regardless of the base rate. If you get a positive result, is it true that P(DPos) = P(CorrectPos) = .95. The paradox lies in the fact that, whatever the base rate, average(Pr(CorrectResult)) = Pr(CorrectPos) x Pr(Pos) + Pr(CorrectNeg) x Pr(Neg) = 0.95. However this does not imply that the components of the average, Pr(CorrectPos) and Pr(CorrectNeg) are themselves equal to 0.95. Depending on the base rate, one could be much lower as long as the other is compensatingly higher.
 Strength of evidence is not strength of effect
 Well documented by Ziliak and McCloskey in The Cult of Statistical Significance.
 Absence of evidence is not evidence of absence
 This is the fallacy that consists in concluding that there no effect if evidence against the null hypothesis does not reach a conventional threshold, e.g. p < 0.05. See, for example, the Vioxx case discussed in Ziliak and McCloskey. Statistical hypothesis testing is not set up to protect again wrongly failing to reject the null. It is set up to control against the possibility of
 Has an interesting history. Donald Rumsfeld refers to this (in?)famously but uses to justify a different conclusion from that for which it is generally used. The usual interpretation is that we cannot prove, and thus maintain uncertainty about, the null hypothesis in the absence of evidence against. Note that this is generally used in the context of frequentist evidence. Rumsfeld uses it to justify the second Iraq war in the absence of evidence that Iraq had weapons of mass destruction. He does not apply the principle to avoid concluding that the null is valid but to justify concluding that it is not.
Teaching topics
 Is buying lottery tickets a good idea?
 Should go deeper than merely considering expectation.
In relation to the test of significance, we may say that a phenomenon is experimentally demonstrable when we know how to conduct an experiment which will rarely fail to give us a statistically significant result.  Fisher 1947
How to teach Bayes
 Gigerenzer (1995): Teaching Bayes with frequency formats (http://www.archwoodside.com/wpcontent/uploads/2015/09/GigerenzerHoffrageHowtoimproveBayesianreasoningwithoutinstruction.pdf)
Miscellaneous Links
 Assessment Resource Tools for Improving Statistical Thinking (https://apps3.cehd.umn.edu/artist/)
 Statistics Education Research Journal (http://iaseweb.org/Publications.php?p=SERJ_issues)
 Rothman and Greeenland (2005) Causation and Causal Inference in Epidemiology (http://www.defendingscience.org/sites/default/files/upload/RothmanGreenland.pdf)
 Greenland, Pearl and Robbins (1999) Causal Diagrams (http://www.biostat.harvard.edu/robins/publications/causaldia.pdf)
 Encyclopedia of Epidemiology, Greenland and Pearl, Causal Diagrams (http://ftp.cs.ucla.edu/pub/stat_ser/r332reprint.pdf)
 NCTM volume on teaching statistics
 Find ASA stuff
 Census at school: discuss limitations
 Teaching Statistics: a bag of tricks
 Resources for MDM4U (https://tapintoteenminds.com/mdm4u/)
 Statistics Education Research Workshop at the Fields Institute, March 11, 2016 (http://www.fields.utoronto.ca/activities/1516/statisticsedresearch)
 Lecture by Andrew Zieffler: Putting the CART before the Horse (http://www.fields.utoronto.ca/videoarchive//event/2096/2016)
 Youtube video on Bayes vs Frequentist inference (https://www.youtube.com/watch?v=BcvLAwJRss&feature=youtu.be)
 Noonan on teaching outcomes (http://www.jeffnoonan.org/?p=2793)
 Regina Nuzzo (2014) "Scientific Method: Statistical Errors", Nature. (http://www.nature.com/news/scientificmethodstatisticalerrors1.14700)
 P values, the 'gold standard' of statistical validity, are not as reliable as many scientists assume.
 Tina A. Grotzer (2012) Learning causality in a complex world : understandings of consequence.
 Project Zero: Causal Cognition in a Complex World (http://www.pz.harvard.edu/projects/causalcognitioninacomplexworld)
 Tina A. Grotzer and David N. Perkins (2000) "A Taxonomy of Causal Models: The Conceptual Leaps Between Models and Students' Reflections on Them". NARST (http://isites.harvard.edu/fs/docs/icb.topic1117424.files/NARSTGrotzerPerkins2000Taxonomy.pdf)
 [Tina A. Grotzer (2003) "Learning to Understand the Forms of Causality Implicit in Scientifically Accepted Explanations", Studies in Science Education, 39:1, 174, DOI:10.1080/03057260308560195]
 Studies in Science Education (http://www.tandfonline.com/loi/rsse20#.Vu7Y2uIrIUE)
 Galit Shmueli (2010) "To Explain or to Predict?" Statistical Science, Vol. 25, No. 3, pp. 289310 (http://www.jstor.org/stable/41058949)
 David N. Perkins & Tina A. Grotzer (2008) "Dimensions of Causal Understanding: The Role of Complex Causal Models in Students' Understanding of Science." Studies in Science Education, 41:1 117165.
 TYM and Alzheimerâ€™s disease: Too many false positives, BMJ 2009 (93% sensitivity and 86% specificity)
Visualization
 What if there were only 100 people on Earth (http://www.core77.com/posts/49281/WhatIfThereWereOnly100PeopleOnEarth)
How to teach ...
R with Markdown
 R packages from Computerworld (http://www.computerworld.com/article/2921176/businessintelligence/greatrpackagesfordataimportwranglingvisualization.html)
Probability
 Review of E. T. Jaynes Probability Theory: The Logic of Science (http://www.ams.org/notices/200601/revfaris.pdf)
 Khan Academy on probability (https://www.khanacademy.org/math/probability/independentdependentprobability/basicprobability/a/probabilitythebasics)
 Online stat book (http://onlinestatbook.com/2/probability/probabilitySA.html)