Should we trust the young Turkers?
‘MTurk may be something of an unknown quantity but it is more diverse than the traditional study pool’
Everyone knows that Amazon turns industries on their heads, from books and ebooks to cloud computing. But most people do not realise that Amazon is also upending social science research, thanks to a service called Amazon Mechanical Turk — often known as MTurk or Turk.
MTurk is an online labour marketplace, originally designed to recruit people to do small tasks that computers couldn’t manage — for example, training a spam filter by categorising emails, deciding whether a photograph matched its description, transcribing audio recordings or flagging adult content. Thanks to MTurk, an employer can hire freelancers to work cheaply on a wide variety of computerised tasks.
Psychologists, behavioural economists and political scientists have now realised that it is potentially far cheaper to pay MTurk workers — “Turkers” — to answer questionnaires and participate in activities than it is to assemble a bespoke online panel or to conduct the research in a laboratory filled with student participants sitting at computers. For just a few dollars and at very short notice, economists can look at competition and co-operation, psychologists can examine the way memory works and political scientists can investigate how our ideology skews our logic. The opportunities are vast and have been swiftly embraced.
“The majority of papers presented at the conferences I go to now use Turk,” says Dan Goldstein, a cognitive psychologist at Microsoft Research. Goldstein, an academic who has also worked at London Business School and Columbia University, has used MTurk in his own research, for instance, into the impact of distracting online display ads.
This stampede to MTurk has made some researchers uneasy. Dan Kahan of Yale Law School studies “motivated reasoning” — the way our goals or political opinions can influence the way we process conflicting evidence. He has written a number of pieces warning about the careless use of the Amazon Turk platform.
The most obvious objection is that Turkers aren’t representative of any particular population one might wish to examine. As an illustration of this, two political scientists hired more than 500 Turkers to complete a very brief survey on the day of the 2012 US presidential election. (Tellingly, the entire survey cost the researchers just $28 and the results arrived within four hours.) The researchers, Sean Richey and Ben Taylor, found that 73 per cent of their Turkers said they had voted for Barack O 12 per cent had voted for “other” — compared with 1.6 per cent of all voters. Mitt Romney polled vastly worse with the Turkers than the US public as a whole. Relative to the general population, Turkers were also more likely to vote and be young, male, poor but highly educated. O it is hard to be sure.
Another objection is that Turkers chat with each other on message boards about the work they’re doing. If a piece of research involves some sort of trick question, they may compare answers. If it measures the ability of workers to co-ordinate without communicating, that communication may be happening anyway through back channels. Researchers may pose clever questions designed to measure personality traits or to probe for logical lapses, not realising the Turkers have seen these questions before and yawn when they roll round again.
Kahan wants academic journals to show more scepticism of MTurk research, and to require researchers to justify their methods in some detail. MTurk may be cheap, says Kahan, but sitting at your desk and thinking through a thought experiment is even cheaper. It doesn’t make either method valid.
Other researchers evidently disagree — perhaps because, in a more purely psychological study, what matters is that Turkers are representatives of the human race rather than of a particular rainbow of American political opinion. Many familiar results from psychology have been replicated on MTurk without trouble.
There’s pragmatism at work here too. After all, the traditional piece of psychological research has been conducted on a small number of undergraduates at Ivy League universities. (Historically, such undergraduates were typically white and male, into the bargain.) MTurk may be something of an unknown quantity but it is more diverse than the traditional study pool. Research can be conducted on a much larger group of subjects, and quickly — no more must researchers wait for students to return after the summer break.
For Dan Goldstein, the downsides are manageable and the advantages enormous. The speed of the research means far more rapid progress and, because MTurk is so cheap, much larger samples can be used. He thinks it’s a big improvement on what went before.
“I think it is one of the most important and beneficial innovations in the history of psychology,” he says, before adding the obvious caveat that like any research tool, it must be wielded with skill.
The original Mechanical Turk was an 18th-century chess-playing “robot” that, in reality, concealed a human chess player. There is something of the Wizard of Oz about the idea — and after a few decades of creating wonderment, the Turk was eventually exposed as nothing more than a clever and seductive trick. For Turk-sceptic Dan Kahan, the analogy is delicious. Having been fooled by a Mechanical Turk once, he says, we should be ashamed to be fooled again.
Written for and first pblished at .
Share this:
Tim Harford is an author, columnist for the Financial Times and presenter of Radio 4's "More or Less".
Tim’s Tweets
Recent Articles
Popular Articles
Free Email Updates
Enter your email address to receive notifications of new articles by email (you can unsubscribe at any time).
Join 3,247 other subscribers
Email Address
Send to Email Address
Your Email Address
Post was not sent - check your email addresses!
Email check failed, please try again
Sorry, your blog cannot share posts by email.您所在位置： &
&nbsp&&nbsp&nbsp&&nbsp
《教学、教科研文献资料荟萃》.doc64页
本文档一共被下载：
次 ,您可全文免费在线阅读后下载本文档。
文档加载中...广告还剩秒
需要金币：200 &&
《教学、教科研文献资料荟萃》.doc
你可能关注的文档：
··········
··········
教学、教科研文献资料荟萃
求学?理科版2014年第1期
本期内容介绍
中国军校大阅兵
你想成为《士兵突击》里袁郎和高城那样的英雄吗？来军校吧！随着我国军事发展的需要，军校已经逐渐形成了一种“4+1”的“合训分流”人才培养模式。“合训”是指学员前四年在综合大学完成本科学历教育合训，“分流”是指学员第五年到指挥院校进行任职培训，毕业可获得工学、军事学两个学士学位。这些...
中华学子劲锐榜
学海无涯，桴浮于海
劲锐人物：郭思敏，武汉大学化学与分子科学学院化学基地班2012级学生。高中毕业于全国名校之一的湖北黄冈中学，高考以620分的佳绩考上武汉大学，获得2012年新生奖学金。 劲锐标志：志在山顶的人，绝不会贪恋山腰的风景。成功的法门不是进攻而是防守，你必须能抵挡住诱惑的干扰，走得毅然决...
学霸炼成记
数学提分，高效则灵
吴郁 高中就读于安徽六安毛坦厂中学，2011年高考成绩为540分，入读东北师范大学数学与统计学院数学与应用数学专业。 安徽六安毛坦厂中学 这里，被广大网友戏称为“亚洲最大的高考机器”“亚洲最大的高考工厂” ――2013年高考考生11222人，9312人过本科线，达线率82.3%，...
流浪在医大
高校小贴士 本期嘉宾：福建医科大学 品 格：勤奋、严谨、求实、创新 创办时间：1937年 学校属性：省属本科重点建设高校 坐 标：福建 优势专业：临床医学、口腔、麻醉 影像护理学、护理学 我的宣言：青春医大 和谐医大 我在流浪，流浪在医大。 福建医科大学在哪？GPS定位系统显示：...
走近高校“不老的音符”
浸润在大学校园的醇厚学风中，高校建筑似乎也多了些与众不同的超凡灵气。那飞
正在加载中，请稍后...君，已阅读到文档的结尾了呢~~
广告剩余8秒
文档加载中
扫扫二维码，随身浏览文档
手机或平板扫扫即可继续访问
Should We Trust Economists？ 经济学家可信吗？.doc
举报该文档为侵权文档。
举报该文档含有违规或不良信息。
反馈该文档无法正常浏览。
举报该文档为重复文档。
推荐理由：
将文档分享至：
分享完整地址
文档地址：
粘贴到BBS或博客
flash地址：
支持嵌入FLASH地址的网站使用
html代码：
&embed src='/DocinViewer--144.swf' width='100%' height='600' type=application/x-shockwave-flash ALLOWFULLSCREEN='true' ALLOWSCRIPTACCESS='always'&&/embed&
450px*300px480px*400px650px*490px
支持嵌入HTML代码的网站使用
您的内容已经提交成功
您所提交的内容需要审核后才能发布，请您等待！
3秒自动关闭窗口试析Should We Trust Economists？ 经济学家可信吗？(2)-7彩论文网
7彩论文网专业提供论文代写与代发表服务其次提供论文范文免费阅读
热门论文关键词
试析Should We Trust Economists？ 经济学家可信吗？(2)
试析Should We Trust Economists？ 经济学家可信吗？(2)
论文导读：解这些基本的想法，并自己决定哪些看法是说得过去的，哪些看法他们认为是不现实的。　　经济学家还有另一个美德，就是他们非常善于指出彼此的逻辑错误。总体而言，经济学家们都是非常聪明、敏锐的人。就像其他所有人那样，他们容易过分自信并过于依赖自己未经证实的理论。但是当他们这么做时，其他经济学家通常会逮他们个正着！因此
lot of smart people have spent a lot of time creating tools with names like “dynamic stochastic general equilibrium11）.” But as of this moment， those models can’t really forecast the economy like our meteorologists12） can forecast the weather. Furthermore， they contain a lot of obviously wrong assumptions. To give just one example， many of Should We Trust Economists？ 学家可信吗？由优秀网站提供,助您写好论文.the models stipulate13） that companies are only allowed to change their prices at random times！ Crazy， right？ Economists include things like that to make the models easier to use， and they hope that those zany14） assumptions are actually decent approximations to the way the world really works. But even with these kludges15） in place， none of the existing models can do much to predict the economy.Should We Trust Economists？ 经济学家可信吗？由优秀论文网站{#GetFullDomain}提供,助您写好论文.Should We Trust Economists？ 经济学家可信吗？由优秀论文网站提供,助您写好论文.顶级的经济学家们十分清楚自己的无知。（美国）联邦储备局主席本?伯南克最近在普林斯顿大学发表了毕业演讲，期间他对听众半开玩笑说：“经济学是一个高度复杂的思想领域，非常擅长向决策者们清晰准确地解释他们过去的选择为什么是错误的，但对于未来，就不那么擅长了。”世界上最著名的家之一格雷格?曼昆在2011年《纽约时报》的专栏中如此表述了这样的感想：　　“在当了超过25年的职业经济学家后，我有件事要坦白：对于经济，我有很多不了解的地方。事实上，研究经济周期起伏的经济学领域是我投入精力最多、关注最多的领域，却也是我发现自己最常遇到一些没有明显答案的重要理由的领域……”　　所有这些都意味着，当一位经济学家告诉你基于某种理论或某个模型推导出的事情时，你应当对此持强烈的怀疑态度。而且该理论或模型越复杂，你就越应该感到可疑。对使用花哨模型的经济学家要多留神。　　如果经济学家确实成功创建出了更奏效的正式模型，那么我们将能够带着理由去找他们（例如“美联储应当印更多的钱吗？”）并完全信任他们的专家倡议。但在那一天到来之前，所有经济学家真正能给我们的是直觉、倡议和想法。就像那位皇家御医一样，我们每个人接下来都不得不自己确定，什么才是我们认为最好的药物。因而当你听经济学家讲话时，关键是尝试去理解他们为什么会产生那种想法。例如，保罗?克鲁格曼认为，在经济萧条期，货币政策的效果并不好，因为名义利率不能降到零以下，而且因为美联储并不总是擅长说服人们相信美联储将在未来允许通货膨胀。罗伯特?巴罗认为没有效果，因为人们预计今天的刺激政策需要未来的税收来买单，于是减少当前的消费以储备资金，用于支付这些未来的税赋。大多数人能够理解这些基本的想法，并自己决定哪些看法是说得过去的，哪些看法他们认为是不现实的。　　经济学家还有另一个美德，就是他们非常善于指出彼此的逻辑错误。总体而言，经济学家们都是非常聪明、敏锐的人。就像其他所有人那样，他们容易过分自信并过于依赖自己未经证实的理论。但是当他们这么做时，其他经济学家通常会逮他们个正着！因此，为了避开过分相信某位经济学家听上去自信的臆断，你应当听听在此理由Should We Trust Economists？ 经济学家可信吗？论文资料由论文网提供,转载请保留地址.上持另一种意见的经济学家有何观点。　　经济学家并不是知道世界如何运转或该如何对其进行精细调整的关键专家，无论我们可能多么希望他们是这样的人。他们不是技工。如果他们像汽车技工那样做事，你应当立即产生怀疑。但是他们的确有很多有趣的意见可以提供。他们可能会帮助你澄清或重新评估你自己关于经济如何运转的看法。他们还能够帮助你发现别人论断中的缺陷。　　最终，你是那位皇家御医。你或许并不是无所不知，但王子奄奄一息，而你得从你已有的“专家”中挑选出一位。　　1.leech [li?t?] n. 水蛭，蚂蟥　　2.humor [?hju?m?（r）] n. （中世纪生理学中所称对人的健康和性情起决定作用的）体液　　3.epistle [??p?sl] n. 书信体诗文　　4.bread mold：[微]面包霉（通常指出现在霉面包等上的黑根霉）　　5.penicillin [?pen??s?l?n] n. [药]青霉素　　6.dungeon [?d?nd??n] n. 地牢　　7.op-ed page：〈美〉（报纸的）专栏版，特写稿版（由专栏作者等署名撰文，与社论版相对）　　8.charlatan [???l?t?n] n. 假装内行的人；冒充者；骗子　　9.crank [kr??k] n. 怪人　　10.for all：虽然，尽管　　11.dynamic sto
全文地址：/jljjxlw/lw558.html
上一论文：
下一论文：
论文写作技巧
关于试析Should We Trust Economists？ 经济学家可信吗？(2)论文范文由7彩论文网整理编辑提供免费阅读}