It's unfortunate that the app was licensed out to a commercial vendor ("Peak Brain Training") when the study received funding by the National Institute for Health Research (via the Department of Health and Social Care) which is taxpayer funded.
Why wasn't the code open sourced and made available under a free license?
I'm disappointed by this too. The FTC fined Peak Brain Training competitor Lumosity $2 million because it "deceived customers about the cognitive and health benefits of its apps and online products."[1] I was more hopeful about the app if it was standalone.
As for the code being open sourced, the results of the research should be publicly available but the material of the research is an asset of the university (like how the computers and beakers used in other experiments wouldn't be given away to the public). Commercialization of research ("tech transfer") involves additional costs and risks that are taken on by research institutions, researchers, and private entities. This PR announcement was likely not coincidentally following the release of the app to the public. There was likely additional costs outside of the original scope of research to make the app robust for public usage outside of the experiment setting.
Personally, I'm disappointed that something that sounds promising may not have a chance to stand on its own as an example of a viable application when other "brain training" apps have shown their more akin to placebos rather.
That's great except Dual N back is, at least for me, impossibly hard to play. And I have in my estimation a good working memory that enables me to work on multiple abstraction levels simultaneously without losing track.
>>> The game involves asking players to watch a series of digits from two to nine flashing up one by one, at a rate of 100 digits per minute. Over the course of five minutes, players must press a button when they start to see a sequence emerge.
C'mon, this could be cloned in a weekend ;)
Publish neuroscience findings in a public access journal by all means. Even including pseudocode of the sequence generation algorithm.
But I am all for distributing their particular implementation via private sector partnerships. That could yield further funding and experimentation. And possibly alleviate taxpayer research dependency for their lab in future.
The work itself taps into some novel neuroscience. Do we we possess a Bayesian brain that estimates probabilities in real time? Altering a belief net based on new evidence. Or are patterns hard wired and must be learned. I think this sort of training game based on integer series could work just as well with text, images, music, video, animation, etc.
You might wish to read Gerd Gigerenzer as well as Keith E. Stanovich, Richard F. West & Maggie E. Toplak before citing Kahneman so quickly, and doubly so in support of such a tremendously strong claim as the one you just made.
Stanovich et al. showed that Kahneman & Tversky's classical "System 1/System 2" dual process model as one too simplistic, outlining at least 3 systems involved:
* The Autonomous Mind
* The Reflective Mind
* The Algorithmic Mind
For points addressed by both Gigerenzer as well as Stanovich et al, see here:
None of this of course directly addresses your "Bayesian brain" point, however, for that, you might want to take a look at "some" articles by psychiatrist Scott Alexander, many of which speculative but citing a lot of sources you might wish to look into:
Real time is also a funny concept when it comes to perception. There is a TED talk where the speaker explains how your perception of "now" is constructed over about half a second after receiving input. And can be retroactively overwritten in the next few seconds, if more convincing input contradicts the created picture. 100 images per seconds seems to be just a notch below this mattering, but probably sustaining this needs concentration.
Research I have seen suggests we do construct Bayesian prices but we don’t apply them rationally. If people assess that something happens 60% of the time, then in a series of 10 observations they will predict it happening only 6 times.
That is to say, they will bet on the 40% probability outcome 4 times, even though they believe the 60% is more likely each time
Could be due to userbase size but, I liked the assessment and decided to purchase one month of pro to see what else was involved. Once I bought pro I did another problem solving focused workout. My percentile on the assessment for this category was 53% before pro now that I'm pro and did one 'workout' It says I'm at 89%... I find that hard to believe.
Edit: Upon further inspection it looks like the percentiles I was reading were for the specific game. It makes sense that for the pay-walled games there aren't as many people playing so it's not a representative sample.
Research funding in the US oddly doesn't seem to provide any rights to the IP to the taxpayers and only limited rights to the government, at least when it comes to the NSF and NIH (DoD research does seem to be different).
You are putting words in my mouth, this isn't what I said.
I called Bayh-Dole act a fascist law, because it encourages the privatization of public research. It is literally a gift from the public to the corporation and encourages public organizations to participate in corporate behavior which is antithetical to their actual stated missions.
The PPP (public private partnerships) in common wealth countries are aimed at achieving
* less oversight of government activity and accountability
* union busting
under the guise of efficiency and cost reductions. And are anti-democratic. Fascism is the description we give to a certain type of corporate-statist behavior. The wonderful (facetious) effect this has is that is imperceptible to a lot of folks while slowly shifting the ideology to the right.
I am not calling people national socialists. I never called a person a fascist. Bayh-Dole was probably well intentioned but the result has been the exclusion of public research for the public good. It stifles more innovation than it encourages. What term would you use?
I've always wondered why grant funded projects are allowed to be patented, do you think just preventing the IP is enough?
I've seen Mariana Mazzucato suggest the government as VC in that they get a portion of the profits from their grant projects, but then I wonder how they could safely regulate their own funding revenue?
I'm no expert in the area of research IP and govt funding, but my interpretation so far has been that the government provides the funding with the expectation that the research will ultimately improve society (even if it is through a for-profit business). It also values the training of graduate students with the funding.
But the opposite happens because the same scientists own IP and run the study to determine its worth. So we know less and waste more than having done nothing.
Allowing the university to get the IP is another way to fund projects. If the university didn't get any of the IP, grants would have to be larger to fund all costs.
Considering the existence of crown copyright, I'm surprised you would expect anything from the UK gov to be open source or similar.
One of my favorite things about the US is that works of the Federal government are public domain. It speaks to the fact the government is representative of the people themselves, not a particular person.
Now granted, universities don't have to release their publicly funded work either in the US, but I think that should be changed.
This goes a little off-topic, but you might be pleasantly surprised about Crown Copyright.
Crown Copyright doesn't deal with university research, only with works done by the government. Pretty much everything released by central government is now licensed through the UK Government Licensing Framework under the Open Government License (OGL), which is essentially CC-BY.
That's good to know! I guess I was a little out of date, I mostly saw this come up when there was a big push to get stuff on en.wikipedia moved to commons, and all of the UK/AU/CA content couldn't be moved for obvious reasons.
Just because the government makes an investment in some research doesn't mean that it has bought the public rights to all commercial applications of that research.
This should be obvious on HN of all places. The government is just an investor in this context. It can demand whatever terms it wants, but the terms you're proposing are basically "100% equity in return for funding initial development costs." Nobody with options would take that deal.
The flipside here is that we're talking about a highly experimental technology that is not trivial to evaluate for consumers, and even likely snake oil (see luminosity)
So the argument to "nobody would take that deal" would be: nobody would make that investment. Lots of tax payer and government money goes to projects that would otherwise not exist, as such their rights and access to information should go beyond what a usual private investor would expect.
This obviously extends to academic freedom as well. If we're going down the road of treating the government as an investor in a 'university entreprise', then the government should obviously have shareholder control over a public university.
There are more than enough researchers and research which have no options, but are still valuable for society. Let the others be done commercially, more public money for those that really need it.
Much publicly funded research never leaves academia and governments want an ROI in terms of seeing the research in products. US research agencies have been trying to incentivize commercialization of research and for PhD students to pursue entrepreneurship. They launched the NSF i-corps program to facilitate that, with the intial curriculum by Steve Blank.
Of course, this article is talking about UK research, but I assume the same desire to have research have impact applies.
Researchers aren't (necessarily) philanthropists. They and the university both get a cut of the IP they develop to encourage them to both invent useful things and to get those useful things into production. If the commercially-viable fruits of this research had to be open-sourced it would be some half-commented github repo that no one would look at twice, because none of the researchers would have any incentive to make it anything more than that.
Haha, I’m just suggesting that where there is a profit to be made human beings tend to engineer opportunities to make cash, or in some cases, tenure.
I would add that this happens everywhere from The Donald and construction of the wall (I’m sure the procurement process will be astonishingly transparent) to government building anything really. Clay Davis (shiiiiiiit) refers to this in The Wire as “the golden faucet”, a mythical place within government that you can make a seemingly infinite amount of money from. Alternatively for the UK read any edition of Private Eye.
well, within capitalism, there is a massive amount of capital trying to find profitable investments of any kind. university research represents a promising source of such investments, so as long as private capital exists and is doing this, there will be all the socioeconomic pressure in the world for it to capture and exploit public research for private profits. sometimes this pressure results in useful things being made and sometimes it results in privatization of healthcare or research, public interest be damned. increasingly over time it is the latter.
The game looks like a gamified version of the test they're using to measure concentration. It seems likely that the subjects are getting better at that specific task through practice, but I'd be skeptical that the results would generalize to other tasks requiring concentration.
Goodhart's law: "When a measure becomes a target, it ceases to be a good measure."
If it is just training to become better at the test to measure ADHD (I don't know enough about the game to be sure) than it would be a classic application of that "law".
I don't know much about the product either, but there are not good quantitative tests to measure ADHD. ADHD is a multifactorial diagnosis that can present with myriad symptoms, and it's diagnosed by examining a list of symptoms and their effect on the patient's life. For instance, do you lose things you need, are you attentive to details, do you daydream or space out to your own detriment, do you fidget or move when it's inappropriate, etc. It'd be very lackluster for a single test, or a single person, to judge these symptoms.
The gold standard ADHD diagnosis would involve examining the child (or adult) in multiple settings with multiple people's evidence, and then comparing symptoms against those for anxiety, depression, and other MH diagnoses. If they have enough symptoms from the official list, and they've been carefully examined, they may qualify. A simple game/test will never be able to replace that process.
One of the big issues with these distress-based diagnosis metrics is that it fails to diagnose people whose social environments are particularly well suited for their mental health issues. If an autistic child has parents who manage their sensory sensitivities and work well with their communicative difficulties, the system sees a child who isn't experiencing a lot of the distress-based symptoms used to diagnose autism.
So like, your gold standard diagnosis ends up depending a lot on the organizational and attention demands the patient is required to meet. A business analyst who is required to sit at a desk for eight hours a day and pay attention to boring numbers on a business spreadsheet is much more likely to get diagnosed with ADHD, while a barber that hangs out and does short haircuts while interacting with clients is much less likely to merit such diagnosis.
Furthermore, many tests for adhd also work as IQ tests, meaning if you have a loq iq you are more likely to get a lower score, and bice versa. I remember one of the tests for adhd involved response time to visual stimuli and how fast a person responded. Response time has a direct correlation to IQ.
I generally agree but there are circumstances where you can just tell. My experience as an adult being rediagnosed was honestly answering a survey of questions. Half way through the psychiatrist already knew it. There was no question.
I was originally diagnosed in college. And have managed it with a combo of effort and medication. Life has been good.
Now I have two family member adolescents. One was diagnosed, and I knew he absolutely did not have it. It took two years before the psychiatrist admitted it was anxiety. Meanwhile the younger one definitely does, but personally I feel he is too young for medication or even be diagnosed. He will eventually be properly diagnosed and be prescribed.
I know there is a lot of misdiagnosis, but sometimes it's incredibly obvious even on a first interaction. I am not a psychiatrist but sometimes you just recognize your people.
Dual N-back is currently the premiere brain-training task, by which I mean it's the only one that's ever looked at all promising for general benefit. Unfortunately, the newest and best studies suggest that even n-back performance is basically just a training effect on a single test, plus possibly improvement in visual processing. (In layman's terms, your memory doesn't improve, but you do get better at carefully watching for a dot.)
This isn't being advertised as more substantive than dual n-back, just more gamified and playable. Given that n-back doesn't seem to work even if you're absolutely diligent about doing it, my expectation for "less rigorous but more playable" is that it will have no value whatsoever.
"To meet the objective, users have to identify different combinations of number strings in missions littered with distraction."
About the test (used to prove the game does improve cognitive abilities):
"Those tested were asked to detect digit sequences (such as 2-4 or 6-8), and then hit a button once detected as quickly as they could — and multiple sequences could appear at the same time"
This feels like a paid article to promote the Peak app. No science here unfortunately. I don't understand how public funds pay for studies like this.
While this may be true, I have found with my own son's that the number one most popular soccer skills test (MacDonald test, 1960) is also a good indication of soccer skills and you can train for it. It's a good adjunct to regular participation in a recreational soccer program. My sons achieved competitive A and B league performance (nerd kids, sons of a PhD computer scientists).
This seems borderline scientific fraud. They are showing that participants get better at a test when they practice the test. They use this to make outlandish claims about the test-training app which is sold by a company previously fined for making fraudulent claims around their braintraining apps.
Pretty much every brain-training routine that's ever showed promise has turned out to be a practice effect specific to the task. "Does it generalize?" is the question for regimens like this. After so many promising failures, I think it's inappropriate to claim even preliminary success without looking at that question. (Worse, in fact: they ran the Trail Making Test too, saw no significant change, and declared success because scores hadn't gone down.)
Unless this stands up to a far stronger test, I'm not just dismissing the result but considering it an embarrassment for Cambridge's Neuroscience Institute, Frontiers in Behavioural Neuroscience, and ABC. Turning the results so far into a news story promising to free people from stimulants is reckless at best and dishonest at worst.
This whole thing has been scrutinized for years. A quick read shows that people that play the game, get better at... playing the game. Whoda thunk? Also, a private company gets to sop up profits from something created by public money. Boo.
It's not exactly encouraging that the first author consults for Peak. The conflict of interest statement in the paper offers this deeply unpleasant comment:
"BS consults for Cambridge Cognition and Peak. We have technology-transferred the App to Cambridge Enterprise who intends to technology-transfer the App to the games company Peak so that it can become widely available for use on mobile devices. This has not occurred yet."
In short: we put up a free demo, but now we're giving all our code to the for-profit company our lead author happens to consult for. Also, they haven't bothered to actually release it outside their branded iOs-only app.
Interesting that their press release claims the “public facing” version of the app is free ... but I can’t find anything except the peak brain trining app, which keeps decoder behind its pay wall.
"a month resulted in neurological improvements in healthy trial participants that were comparable to those taking stimulants such as methylphenidate (Ritalin) or nicotine."
I was pretty excited to read this headline, but the details don't make sense to me. They only did this on people without ADD?
I don't understand why they didn't test this on people who have ADHD. "Healthy trial participants" would never be taking stimulants such as methylphenidate (Ritalin), so how do they compare how a healthy person changes from Ritalin.
> "Healthy trial participants" would never be taking stimulants such as methylphenidate (Ritalin), so how do they compare how a healthy person changes from Ritalin.
Maybe in your circles. But I know a lot of people who abuse focus enhancing drugs (often touted as "nootropics") to keep up with their academic struggles.
I think the point is that there is little research on how healthy patients react to stimulants because they don't need them in the same way that those with ADHD do. This study says nothing about how this may help people with ADHD because you can't assume that they will have the same result.
> I don't understand why they didn't test this on people who have ADHD.
Ease of sample, probably, but also as a baseline proof. There's essentially no evidence of any brain-training game ever producing significant cognitive improvements in populations with roughly normal functioning (in which I'm including ADHD). So before pitching it as a Ritalin replacement, step one would be to demonstrate that it achieves anything at all.
Unfortunately, they don't seem to have done that either, because the test was so close to the game that practice effects are the most likely explanation.
I didn't read the entire study and didn't play too much with the app, but this entire article reads like an advertisement to the peak app. The only relation I found on peak's about page to Cambridge is Barbara Sahakian, FMedSci DSc, Scientific Advisor. http://www.peak.net/about/
I'm not calling this a fraud but didn't numerous studies showed that any of the so called brain trainig apps don't show translation of skill to the real world? e.g. you do get better that the games the apps have, but not helping in actual real life scenarios?
The test they used (Rapid Visual Information Processing test) is just another "game" in a way. You can train someone to be better at the test, but it doesn't make their ADHD go away magically. I wish it was that easy.
It looks like the app includes games that are way too similar to the actual test that was used to determine progress in the test subjects.
From the article:
About the game:
"To meet the objective, users have to identify different combinations of number strings in missions littered with distraction."
About the test (used to prove the game does improve cognitive abilities):
"Those tested were asked to detect digit sequences (such as 2-4 or 6-8), and then hit a button once detected as quickly as they could — and multiple sequences could appear at the same time"
Am I missing something or this sounds like almost the same thing?
One can train and learn all the answers to an IQ test, this doesn't make one a genius.
This feels like a real great promoted article in the cover of scientific writing, but maybe I'm just and old cynic.
Once a study shows actual translation, after years of observation, that ADHD subjects get improvements in other aspects of life (school, social, grades, self esteem etc) then I'll be the first to support it. Until then, it just looks like greed and bottom line. I guess it's better than playing candy crush at least.
> didn't numerous studies showed that any of the so called brain trainig apps don't show translation of skill to the real world? e.g. you do get better that the games the apps have, but not helping in actual real life scenarios?
Yes, this is almost always the problem. For quite a while, we thought dual n-back tests had generalized results, and so brain-training apps were basically attempting to stumble on a gamified version of that task. Unfortunately, newer studies[1] suggest that dual n-back also doesn't produce fluid intelligence or working memory improvements, so there's no proof of concept here at all.
As for this study, the game and metric were exceedingly similar, and practice effects on the metric are an obvious risk. The researchers did apply a second metric less similar to their task, a standard dot-connecting task called Trail Making Task. Unfortunately, their stated intent was to show that improved single-task attention didn't degrade focus-switching attention, so this was a substantially different task for which they hypothesized no change.
Their abstract reports a statistically significant improvement on TMT, which could be quite interesting. However, the full paper shows that Decoder had p=0.03 improvement on TMT over the active control (people who played Bingo), but the passive control (no play) was not significantly different than Decoder or the active control. This isn't regarded as a failure because they expected no change, but it means the study failed to show clear improvement in any metric which didn't precisely replicate the game's task.
You're absolutely right to be skeptical here. Practice effects are a constant problem with this sort of research, and the study seems to be confusingly short on attempts to test for that. The actual app focuses more on gamification than using any type of proven test, and the lead author's corporate connection to Peak is at best concerning. I don't think there's any sort of fraud here, but it's a field riddled with well-meaning projects that turn out to lack any general benefit, and this result has all the warning signs for another promising failure.
I understand if there's no web version, but no Android version in 2019 is ridiculous. That and the plus university research angle seems to indicate that this is a one-off app that is unlikely to be updated or maintained down the line.
I think it's the opposite problem, actually: Decoder is made by Peak, a company that specializes in Lumosity-style brain training games. This is basically a corporate/academic partnership, where Peak just wanted to develop for the higher-margin platform.
As for the university research angle... it's bad research, embarrassingly so. There's a reason companies like Lumosity keep getting warned about making unsupported claims, and this looks like an attempt to get academic backing for a product with all the same problems as prior ones.
I always thought of Ritalin as a thing for kids. Then I went to law school on the east coast of the US. What totally changed my perspective was the large number of adult students (mid-20s/30s) still on Ritalin. I was no slouch, but many of them were getting better grades than me. At what point did Ritalin shift from being something to correct a legitimate deficiency into something meant to increase performance? If you are getting strait As in a graduate program, maybe you don't need it as much as you think.
> I was no slouch, but many of them were getting better grades than me.
ADHD doesn't mean stupid. Many people who suffer under ADHD are really bright, they just have problems with focusing in school.
And AFAIK Ritalin doesn't increase intelligence, the best explanation I've come across is it just lowers the threshold for "interesting" in the ADHD brain until things like driving according to speed limits and listening to teachers isn't mind numbingly boring anymore.
Very interesting to see how a particular style of game can have beneficial effects on attention. I wonder if the opposite is also true, although I'm sure there's some ethical issue with trying to find out.
So basically this is a counter study to the 'brain stimulating games' do nothing papers.
Presumably the game is not that complex in its mechanic such that an open source version that ran on more platforms could be devised.
That said, I'm waiting for the study where they take participants and structure their access to email, social media, and television in such a way as to limit it to a prescribed time and duration. One hypothesis is that trying to multi-task negatively affects the ability to single task (concentrate) effectively.
This is pretty cool. When I was younger I was hooked up to a game that was similar to EKGs with the connectors on my head and I would have to concentrate to play the game. It's great seeing its available for anyone with a phone.
I think VR games would be great for improving concentration, but at short intervals at first. Similar to meditation where you focus on breathing and then move onto other areas of meditation.
I wonder if they did research into MOBA games like League of Legends before making this app. League of Legends and other MOBA games are fun, competitive and require a boatload of concentration and reflexes if you wanna get good. I would point people to play those games if they wanna improve their concentration, although I'm not sure how much research there is on it.
The reward mechanism in games is very different from real life. It is very immediate and accurate and that's why so many , especially men, enjoy doing it. It doesn't really help with anything else.
ABC News? Probably one of the most reputable news sources in Australia. It's also funded and owned by the government and has no ads so I doubt it's paid advertising.
Anecdotally, for awhile I was solving Soduku entirely in my head, and I had a noticeable improvement in memory. Once I stopped the improvement went away. So, there seems to be benefit to some sorts of brain training.
I memorize the layout, and solved the puzzle by imagining the grid. Don't use paper at all. The easy puzzles tend to have a certain symmetry that make them not too difficult to memorize.
Cambridge is a city. There are many organisations calling themselves Cambridge (or Oxford for that matter), either because they are genuinely based there or because they hope to benefit from the association. Can't really trademark a city name though.
Let’s not forget that it was Cambridge who first identified the political opportunities of knowing people’s psychological tendencies through survey collection on Facebook - before Cambridge Analytica was born.
Can’t help but feel that this is slightly on the nose.
Let’s not forget that it was Cambridge who first identified the political opportunities of knowing people’s psychological tendencies through survey collection on Facebook - before Cambridge Analytica was born.
Can’t help but feel that doing something similar with cognition is slightly on the nose.
It's unfortunate that the app was licensed out to a commercial vendor ("Peak Brain Training") when the study received funding by the National Institute for Health Research (via the Department of Health and Social Care) which is taxpayer funded.
Why wasn't the code open sourced and made available under a free license?
I'm disappointed by this too. The FTC fined Peak Brain Training competitor Lumosity $2 million because it "deceived customers about the cognitive and health benefits of its apps and online products."[1] I was more hopeful about the app if it was standalone.
As for the code being open sourced, the results of the research should be publicly available but the material of the research is an asset of the university (like how the computers and beakers used in other experiments wouldn't be given away to the public). Commercialization of research ("tech transfer") involves additional costs and risks that are taken on by research institutions, researchers, and private entities. This PR announcement was likely not coincidentally following the release of the app to the public. There was likely additional costs outside of the original scope of research to make the app robust for public usage outside of the experiment setting.
Personally, I'm disappointed that something that sounds promising may not have a chance to stand on its own as an example of a viable application when other "brain training" apps have shown their more akin to placebos rather.
There's no reason to believe that this new app is any less snake oil than Lumosity
Its story has the same trajectory
I agree. Summary of findings:
“Test subjects who spent hours practicing [insert proprietary game here] scored better when tested on different games that require similar skills.”
Smells like a poorly designed experiment salted with commercial interests.
Yup. A similar thing happened with a training/game called the "N-back" test that allegedly could improve executive function and IQ!
Turned out the test of IQ was Raven's Progressive Matrices and it's never been decisively validated
https://en.wikipedia.org/wiki/N-back#Use_in_tutoring_and_reh...
https://hn.algolia.com/?query=n-back&sort=byPopularity&prefi...
Dual N-back is reported to improve working memory, which can help improve IQ. A excellent report is here https://www.gwern.net/DNB-FAQ
That's great except Dual N back is, at least for me, impossibly hard to play. And I have in my estimation a good working memory that enables me to work on multiple abstraction levels simultaneously without losing track.
Are there any tips on how to not miserably fail?
That's vague. Do you have specific examples of flaws in methodology or analysis in the published peer reviewed results here [1]?
[1] https://www.frontiersin.org/articles/10.3389/fnbeh.2019.0000...
Very cool analogy about the beakers and equipment. Something I never considered.
>>> The game involves asking players to watch a series of digits from two to nine flashing up one by one, at a rate of 100 digits per minute. Over the course of five minutes, players must press a button when they start to see a sequence emerge.
C'mon, this could be cloned in a weekend ;)
Publish neuroscience findings in a public access journal by all means. Even including pseudocode of the sequence generation algorithm.
But I am all for distributing their particular implementation via private sector partnerships. That could yield further funding and experimentation. And possibly alleviate taxpayer research dependency for their lab in future.
The work itself taps into some novel neuroscience. Do we we possess a Bayesian brain that estimates probabilities in real time? Altering a belief net based on new evidence. Or are patterns hard wired and must be learned. I think this sort of training game based on integer series could work just as well with text, images, music, video, animation, etc.
> Do we possess a Bayesian brain that estimates probabilities in real time?
You should read Daniel Kahneman's "Thinking Fast and Slow" - the answer is a resounding "No".
You might wish to read Gerd Gigerenzer as well as Keith E. Stanovich, Richard F. West & Maggie E. Toplak before citing Kahneman so quickly, and doubly so in support of such a tremendously strong claim as the one you just made.
For Gigerenzer, you might wish to start out here:
https://en.wikipedia.org/wiki/Gerd_Gigerenzer#Heuristics
https://www.tandfonline.com/doi/abs/10.1080/1479277914300003... (Available here: http://library.mpib-berlin.mpg.de/ft/gg/gg_how_1991.pdf)
And for Stanovich et al., you might want to start out here:
http://cognet.mit.edu/book/rationality-quotient
https://en.wikipedia.org/wiki/Dysrationalia
Stanovich et al. showed that Kahneman & Tversky's classical "System 1/System 2" dual process model as one too simplistic, outlining at least 3 systems involved:
* The Autonomous Mind
* The Reflective Mind
* The Algorithmic Mind
For points addressed by both Gigerenzer as well as Stanovich et al, see here:
https://en.wikipedia.org/wiki/Great_Rationality_Debate
None of this of course directly addresses your "Bayesian brain" point, however, for that, you might want to take a look at "some" articles by psychiatrist Scott Alexander, many of which speculative but citing a lot of sources you might wish to look into:
https://www.lesswrong.com/posts/ZiQqsgGX6a42Sfpii/the-apolog...
https://www.lesswrong.com/posts/euJm4RwkAptZnP89i/bayes-for-...
https://slatestarcodex.com/2015/08/28/mysticism-and-pattern-...
http://slatestarcodex.com/2016/09/12/its-bayes-all-the-way-u...
http://slatestarcodex.com/2017/06/28/why-are-transgender-peo...
http://slatestarcodex.com/2018/12/11/diametrical-model-of-au...
https://slatestarcodex.com/2019/01/10/paradigms-all-the-way-...
(And here, as a bonus, an article on dreaming, by Eliezer Yudkowsky: https://www.lesswrong.com/posts/8z2Fm2yaHpQz8rr5B/dreams-wit...)
Real time is also a funny concept when it comes to perception. There is a TED talk where the speaker explains how your perception of "now" is constructed over about half a second after receiving input. And can be retroactively overwritten in the next few seconds, if more convincing input contradicts the created picture. 100 images per seconds seems to be just a notch below this mattering, but probably sustaining this needs concentration.
Research I have seen suggests we do construct Bayesian prices but we don’t apply them rationally. If people assess that something happens 60% of the time, then in a series of 10 observations they will predict it happening only 6 times.
That is to say, they will bet on the 40% probability outcome 4 times, even though they believe the 60% is more likely each time
It took me a second to realize the meaning of this; do you have any links - sounds fascinating :)
Could be due to userbase size but, I liked the assessment and decided to purchase one month of pro to see what else was involved. Once I bought pro I did another problem solving focused workout. My percentile on the assessment for this category was 53% before pro now that I'm pro and did one 'workout' It says I'm at 89%... I find that hard to believe.
Edit: Upon further inspection it looks like the percentiles I was reading were for the specific game. It makes sense that for the pay-walled games there aren't as many people playing so it's not a representative sample.
Research funding in the US oddly doesn't seem to provide any rights to the IP to the taxpayers and only limited rights to the government, at least when it comes to the NSF and NIH (DoD research does seem to be different).
Disclaimer: I'm a researcher in the US.
https://en.m.wikipedia.org/wiki/Bayh–Dole_Act
Is what gives universities IP rights and allows them to keep research closed. Top Five Fascist laws in the books.
Fascist seems like a pretty extreme overreaction.
Corporatized public institutions are clearly a fascist move. Same goes for the public private partnerships in common wealth countries.
So anyone who implemented a PPI in the MHS is a fascist? Great way to devalue the term.
You are putting words in my mouth, this isn't what I said.
I called Bayh-Dole act a fascist law, because it encourages the privatization of public research. It is literally a gift from the public to the corporation and encourages public organizations to participate in corporate behavior which is antithetical to their actual stated missions.
The PPP (public private partnerships) in common wealth countries are aimed at achieving
* less oversight of government activity and accountability
* union busting
under the guise of efficiency and cost reductions. And are anti-democratic. Fascism is the description we give to a certain type of corporate-statist behavior. The wonderful (facetious) effect this has is that is imperceptible to a lot of folks while slowly shifting the ideology to the right.
I am not calling people national socialists. I never called a person a fascist. Bayh-Dole was probably well intentioned but the result has been the exclusion of public research for the public good. It stifles more innovation than it encourages. What term would you use?
I've always wondered why grant funded projects are allowed to be patented, do you think just preventing the IP is enough?
I've seen Mariana Mazzucato suggest the government as VC in that they get a portion of the profits from their grant projects, but then I wonder how they could safely regulate their own funding revenue?
I'm no expert in the area of research IP and govt funding, but my interpretation so far has been that the government provides the funding with the expectation that the research will ultimately improve society (even if it is through a for-profit business). It also values the training of graduate students with the funding.
But the opposite happens because the same scientists own IP and run the study to determine its worth. So we know less and waste more than having done nothing.
Allowing the university to get the IP is another way to fund projects. If the university didn't get any of the IP, grants would have to be larger to fund all costs.
Considering the existence of crown copyright, I'm surprised you would expect anything from the UK gov to be open source or similar.
One of my favorite things about the US is that works of the Federal government are public domain. It speaks to the fact the government is representative of the people themselves, not a particular person.
Now granted, universities don't have to release their publicly funded work either in the US, but I think that should be changed.
This goes a little off-topic, but you might be pleasantly surprised about Crown Copyright.
Crown Copyright doesn't deal with university research, only with works done by the government. Pretty much everything released by central government is now licensed through the UK Government Licensing Framework under the Open Government License (OGL), which is essentially CC-BY.
See http://www.nationalarchives.gov.uk/doc/open-government-licen...
Examples of loads of photos and documents that Wikimedia have taken copies of (and often reusing) under the OGL: https://commons.wikimedia.org/wiki/Category:OGL
That's good to know! I guess I was a little out of date, I mostly saw this come up when there was a big push to get stuff on en.wikipedia moved to commons, and all of the UK/AU/CA content couldn't be moved for obvious reasons.
Canada has their own version of the OGL too. Dunno about Australia.
https://open.canada.ca/en/open-government-licence-canada
One need not have an expectation of a good outcome to be disappointed that it doesn't happen.
Just because the government makes an investment in some research doesn't mean that it has bought the public rights to all commercial applications of that research.
This should be obvious on HN of all places. The government is just an investor in this context. It can demand whatever terms it wants, but the terms you're proposing are basically "100% equity in return for funding initial development costs." Nobody with options would take that deal.
The flipside here is that we're talking about a highly experimental technology that is not trivial to evaluate for consumers, and even likely snake oil (see luminosity)
So the argument to "nobody would take that deal" would be: nobody would make that investment. Lots of tax payer and government money goes to projects that would otherwise not exist, as such their rights and access to information should go beyond what a usual private investor would expect.
This obviously extends to academic freedom as well. If we're going down the road of treating the government as an investor in a 'university entreprise', then the government should obviously have shareholder control over a public university.
> Nobody with options would take that deal.
There are more than enough researchers and research which have no options, but are still valuable for society. Let the others be done commercially, more public money for those that really need it.
Definitely. Public funding should come with proper licensing restrictions that force open source.
Much publicly funded research never leaves academia and governments want an ROI in terms of seeing the research in products. US research agencies have been trying to incentivize commercialization of research and for PhD students to pursue entrepreneurship. They launched the NSF i-corps program to facilitate that, with the intial curriculum by Steve Blank.
Of course, this article is talking about UK research, but I assume the same desire to have research have impact applies.
Isn't that a little like saying that the government expects to see a "ROI" from educating children?
The positive effects of open sourcing science could be seen down the road and not in a direct way, but in more indirect ways.
Governments do expect an ROI from educating children. Educated children make employable adults who make taxable income which makes governments happy.
Researchers aren't (necessarily) philanthropists. They and the university both get a cut of the IP they develop to encourage them to both invent useful things and to get those useful things into production. If the commercially-viable fruits of this research had to be open-sourced it would be some half-commented github repo that no one would look at twice, because none of the researchers would have any incentive to make it anything more than that.
Thanks for Peak Brain that app is an ad hell hole.
Because capitalism?
I'm not sure I understand could you go into more detail?
Haha, I’m just suggesting that where there is a profit to be made human beings tend to engineer opportunities to make cash, or in some cases, tenure.
I would add that this happens everywhere from The Donald and construction of the wall (I’m sure the procurement process will be astonishingly transparent) to government building anything really. Clay Davis (shiiiiiiit) refers to this in The Wire as “the golden faucet”, a mythical place within government that you can make a seemingly infinite amount of money from. Alternatively for the UK read any edition of Private Eye.
well, within capitalism, there is a massive amount of capital trying to find profitable investments of any kind. university research represents a promising source of such investments, so as long as private capital exists and is doing this, there will be all the socioeconomic pressure in the world for it to capture and exploit public research for private profits. sometimes this pressure results in useful things being made and sometimes it results in privatization of healthcare or research, public interest be damned. increasingly over time it is the latter.
The game looks like a gamified version of the test they're using to measure concentration. It seems likely that the subjects are getting better at that specific task through practice, but I'd be skeptical that the results would generalize to other tasks requiring concentration.
Goodhart's law: "When a measure becomes a target, it ceases to be a good measure."
If it is just training to become better at the test to measure ADHD (I don't know enough about the game to be sure) than it would be a classic application of that "law".
I don't know much about the product either, but there are not good quantitative tests to measure ADHD. ADHD is a multifactorial diagnosis that can present with myriad symptoms, and it's diagnosed by examining a list of symptoms and their effect on the patient's life. For instance, do you lose things you need, are you attentive to details, do you daydream or space out to your own detriment, do you fidget or move when it's inappropriate, etc. It'd be very lackluster for a single test, or a single person, to judge these symptoms.
The gold standard ADHD diagnosis would involve examining the child (or adult) in multiple settings with multiple people's evidence, and then comparing symptoms against those for anxiety, depression, and other MH diagnoses. If they have enough symptoms from the official list, and they've been carefully examined, they may qualify. A simple game/test will never be able to replace that process.
One of the big issues with these distress-based diagnosis metrics is that it fails to diagnose people whose social environments are particularly well suited for their mental health issues. If an autistic child has parents who manage their sensory sensitivities and work well with their communicative difficulties, the system sees a child who isn't experiencing a lot of the distress-based symptoms used to diagnose autism.
So like, your gold standard diagnosis ends up depending a lot on the organizational and attention demands the patient is required to meet. A business analyst who is required to sit at a desk for eight hours a day and pay attention to boring numbers on a business spreadsheet is much more likely to get diagnosed with ADHD, while a barber that hangs out and does short haircuts while interacting with clients is much less likely to merit such diagnosis.
Furthermore, many tests for adhd also work as IQ tests, meaning if you have a loq iq you are more likely to get a lower score, and bice versa. I remember one of the tests for adhd involved response time to visual stimuli and how fast a person responded. Response time has a direct correlation to IQ.
I generally agree but there are circumstances where you can just tell. My experience as an adult being rediagnosed was honestly answering a survey of questions. Half way through the psychiatrist already knew it. There was no question.
I was originally diagnosed in college. And have managed it with a combo of effort and medication. Life has been good.
Now I have two family member adolescents. One was diagnosed, and I knew he absolutely did not have it. It took two years before the psychiatrist admitted it was anxiety. Meanwhile the younger one definitely does, but personally I feel he is too young for medication or even be diagnosed. He will eventually be properly diagnosed and be prescribed.
I know there is a lot of misdiagnosis, but sometimes it's incredibly obvious even on a first interaction. I am not a psychiatrist but sometimes you just recognize your people.
Dual N-back is currently the premiere brain-training task, by which I mean it's the only one that's ever looked at all promising for general benefit. Unfortunately, the newest and best studies suggest that even n-back performance is basically just a training effect on a single test, plus possibly improvement in visual processing. (In layman's terms, your memory doesn't improve, but you do get better at carefully watching for a dot.)
This isn't being advertised as more substantive than dual n-back, just more gamified and playable. Given that n-back doesn't seem to work even if you're absolutely diligent about doing it, my expectation for "less rigorous but more playable" is that it will have no value whatsoever.
https://www.gwern.net/docs/dnb/2013-colom.pdf
Exactly.
About the game:
"To meet the objective, users have to identify different combinations of number strings in missions littered with distraction."
About the test (used to prove the game does improve cognitive abilities):
"Those tested were asked to detect digit sequences (such as 2-4 or 6-8), and then hit a button once detected as quickly as they could — and multiple sequences could appear at the same time"
This feels like a paid article to promote the Peak app. No science here unfortunately. I don't understand how public funds pay for studies like this.
While this may be true, I have found with my own son's that the number one most popular soccer skills test (MacDonald test, 1960) is also a good indication of soccer skills and you can train for it. It's a good adjunct to regular participation in a recreational soccer program. My sons achieved competitive A and B league performance (nerd kids, sons of a PhD computer scientists).
This seems borderline scientific fraud. They are showing that participants get better at a test when they practice the test. They use this to make outlandish claims about the test-training app which is sold by a company previously fined for making fraudulent claims around their braintraining apps.
Pretty much every brain-training routine that's ever showed promise has turned out to be a practice effect specific to the task. "Does it generalize?" is the question for regimens like this. After so many promising failures, I think it's inappropriate to claim even preliminary success without looking at that question. (Worse, in fact: they ran the Trail Making Test too, saw no significant change, and declared success because scores hadn't gone down.)
Unless this stands up to a far stronger test, I'm not just dismissing the result but considering it an embarrassment for Cambridge's Neuroscience Institute, Frontiers in Behavioural Neuroscience, and ABC. Turning the results so far into a news story promising to free people from stimulants is reckless at best and dishonest at worst.
If you look at the study they don't actually test anyone with ADHD at all. Just talk about it.
They should have a population with confirmed ADHD taje Ritalin or not and then play the game. The headline had nothing to do with the drug or ADHD.
People who played more got better compared to playing bingo is all they could take claim.
This whole thing has been scrutinized for years. A quick read shows that people that play the game, get better at... playing the game. Whoda thunk? Also, a private company gets to sop up profits from something created by public money. Boo.
It's not exactly encouraging that the first author consults for Peak. The conflict of interest statement in the paper offers this deeply unpleasant comment:
"BS consults for Cambridge Cognition and Peak. We have technology-transferred the App to Cambridge Enterprise who intends to technology-transfer the App to the games company Peak so that it can become widely available for use on mobile devices. This has not occurred yet."
In short: we put up a free demo, but now we're giving all our code to the for-profit company our lead author happens to consult for. Also, they haven't bothered to actually release it outside their branded iOs-only app.
This is pretty typical in the psychiatry and psychology field. The only way to make big money is to write a book or invent a test.
Interesting that their press release claims the “public facing” version of the app is free ... but I can’t find anything except the peak brain trining app, which keeps decoder behind its pay wall.
Thanks. You saved me wasting time downloading it.
And creating an account just to do anything. And being subject to whatever horrors lie in their privacy policy.
Wonder how it compares to the dual n-back exercise. https://en.wikipedia.org/wiki/N-back
"a month resulted in neurological improvements in healthy trial participants that were comparable to those taking stimulants such as methylphenidate (Ritalin) or nicotine."
I was pretty excited to read this headline, but the details don't make sense to me. They only did this on people without ADD?
I don't understand why they didn't test this on people who have ADHD. "Healthy trial participants" would never be taking stimulants such as methylphenidate (Ritalin), so how do they compare how a healthy person changes from Ritalin.
> "Healthy trial participants" would never be taking stimulants such as methylphenidate (Ritalin), so how do they compare how a healthy person changes from Ritalin.
Maybe in your circles. But I know a lot of people who abuse focus enhancing drugs (often touted as "nootropics") to keep up with their academic struggles.
I think the point is that there is little research on how healthy patients react to stimulants because they don't need them in the same way that those with ADHD do. This study says nothing about how this may help people with ADHD because you can't assume that they will have the same result.
Yeah, there’s a pretty thriving resale market for Ritalin and Adderall at any given college.
> I don't understand why they didn't test this on people who have ADHD.
Ease of sample, probably, but also as a baseline proof. There's essentially no evidence of any brain-training game ever producing significant cognitive improvements in populations with roughly normal functioning (in which I'm including ADHD). So before pitching it as a Ritalin replacement, step one would be to demonstrate that it achieves anything at all.
Unfortunately, they don't seem to have done that either, because the test was so close to the game that practice effects are the most likely explanation.
I didn't read the entire study and didn't play too much with the app, but this entire article reads like an advertisement to the peak app. The only relation I found on peak's about page to Cambridge is Barbara Sahakian, FMedSci DSc, Scientific Advisor. http://www.peak.net/about/
I'm not calling this a fraud but didn't numerous studies showed that any of the so called brain trainig apps don't show translation of skill to the real world? e.g. you do get better that the games the apps have, but not helping in actual real life scenarios?
The test they used (Rapid Visual Information Processing test) is just another "game" in a way. You can train someone to be better at the test, but it doesn't make their ADHD go away magically. I wish it was that easy.
It looks like the app includes games that are way too similar to the actual test that was used to determine progress in the test subjects.
From the article:
About the game:
"To meet the objective, users have to identify different combinations of number strings in missions littered with distraction."
About the test (used to prove the game does improve cognitive abilities):
"Those tested were asked to detect digit sequences (such as 2-4 or 6-8), and then hit a button once detected as quickly as they could — and multiple sequences could appear at the same time"
Am I missing something or this sounds like almost the same thing?
One can train and learn all the answers to an IQ test, this doesn't make one a genius.
This feels like a real great promoted article in the cover of scientific writing, but maybe I'm just and old cynic.
Once a study shows actual translation, after years of observation, that ADHD subjects get improvements in other aspects of life (school, social, grades, self esteem etc) then I'll be the first to support it. Until then, it just looks like greed and bottom line. I guess it's better than playing candy crush at least.
> didn't numerous studies showed that any of the so called brain trainig apps don't show translation of skill to the real world? e.g. you do get better that the games the apps have, but not helping in actual real life scenarios?
Yes, this is almost always the problem. For quite a while, we thought dual n-back tests had generalized results, and so brain-training apps were basically attempting to stumble on a gamified version of that task. Unfortunately, newer studies[1] suggest that dual n-back also doesn't produce fluid intelligence or working memory improvements, so there's no proof of concept here at all.
As for this study, the game and metric were exceedingly similar, and practice effects on the metric are an obvious risk. The researchers did apply a second metric less similar to their task, a standard dot-connecting task called Trail Making Task. Unfortunately, their stated intent was to show that improved single-task attention didn't degrade focus-switching attention, so this was a substantially different task for which they hypothesized no change.
Their abstract reports a statistically significant improvement on TMT, which could be quite interesting. However, the full paper shows that Decoder had p=0.03 improvement on TMT over the active control (people who played Bingo), but the passive control (no play) was not significantly different than Decoder or the active control. This isn't regarded as a failure because they expected no change, but it means the study failed to show clear improvement in any metric which didn't precisely replicate the game's task.
You're absolutely right to be skeptical here. Practice effects are a constant problem with this sort of research, and the study seems to be confusingly short on attempts to test for that. The actual app focuses more on gamification than using any type of proven test, and the lead author's corporate connection to Peak is at best concerning. I don't think there's any sort of fraud here, but it's a field riddled with well-meaning projects that turn out to lack any general benefit, and this result has all the warning signs for another promising failure.
[1] https://www.sciencedirect.com/science/article/pii/S016028961...
> available on iPhone and iPad
I understand if there's no web version, but no Android version in 2019 is ridiculous. That and the plus university research angle seems to indicate that this is a one-off app that is unlikely to be updated or maintained down the line.
I think it's the opposite problem, actually: Decoder is made by Peak, a company that specializes in Lumosity-style brain training games. This is basically a corporate/academic partnership, where Peak just wanted to develop for the higher-margin platform.
As for the university research angle... it's bad research, embarrassingly so. There's a reason companies like Lumosity keep getting warned about making unsupported claims, and this looks like an attempt to get academic backing for a product with all the same problems as prior ones.
I always thought of Ritalin as a thing for kids. Then I went to law school on the east coast of the US. What totally changed my perspective was the large number of adult students (mid-20s/30s) still on Ritalin. I was no slouch, but many of them were getting better grades than me. At what point did Ritalin shift from being something to correct a legitimate deficiency into something meant to increase performance? If you are getting strait As in a graduate program, maybe you don't need it as much as you think.
> I was no slouch, but many of them were getting better grades than me.
ADHD doesn't mean stupid. Many people who suffer under ADHD are really bright, they just have problems with focusing in school.
And AFAIK Ritalin doesn't increase intelligence, the best explanation I've come across is it just lowers the threshold for "interesting" in the ADHD brain until things like driving according to speed limits and listening to teachers isn't mind numbingly boring anymore.
ADHD is a genetic issue related (in most cases) to dopamine pathways in the brain. You don't grow out of it, just like you don't grow out of diabetes.
https://www.youtube.com/watch?v=jhcn1_qsYmg
It's both. The latter isn't the intended use, of course.
Very interesting to see how a particular style of game can have beneficial effects on attention. I wonder if the opposite is also true, although I'm sure there's some ethical issue with trying to find out.
Does the app block access to Hacker News
and reddit and youtube?
In all seriousness I downloaded a browser extension to lock me into my course work when I need it. It's pretty sad for me
So basically this is a counter study to the 'brain stimulating games' do nothing papers.
Presumably the game is not that complex in its mechanic such that an open source version that ran on more platforms could be devised.
That said, I'm waiting for the study where they take participants and structure their access to email, social media, and television in such a way as to limit it to a prescribed time and duration. One hypothesis is that trying to multi-task negatively affects the ability to single task (concentrate) effectively.
This is pretty cool. When I was younger I was hooked up to a game that was similar to EKGs with the connectors on my head and I would have to concentrate to play the game. It's great seeing its available for anyone with a phone.
I think VR games would be great for improving concentration, but at short intervals at first. Similar to meditation where you focus on breathing and then move onto other areas of meditation.
I wonder if they did research into MOBA games like League of Legends before making this app. League of Legends and other MOBA games are fun, competitive and require a boatload of concentration and reflexes if you wanna get good. I would point people to play those games if they wanna improve their concentration, although I'm not sure how much research there is on it.
The reward mechanism in games is very different from real life. It is very immediate and accurate and that's why so many , especially men, enjoy doing it. It doesn't really help with anything else.
Whomst?
The article points to Decoder being embedded in an app, Peak Brain training:
https://itunes.apple.com/gb/app/peak-brain-training/id806223...
From the article it’s twice as effective as playing bingo, I take this is the FDA approved standard bingo, quite a strange comparison to make.
Bingo has been studied for its effects in this regard: https://journals.sagepub.com/doi/abs/10.1177/153331750101600...
That site is not a news site... That's just paid advertising?
ABC News? Probably one of the most reputable news sources in Australia. It's also funded and owned by the government and has no ads so I doubt it's paid advertising.
Sounds very much like the snake oil that accompanied Lumosity...
Anecdotally, for awhile I was solving Soduku entirely in my head, and I had a noticeable improvement in memory. Once I stopped the improvement went away. So, there seems to be benefit to some sorts of brain training.
How do you solve soduku entirely in your head?
I memorize the layout, and solved the puzzle by imagining the grid. Don't use paper at all. The easy puzzles tend to have a certain symmetry that make them not too difficult to memorize.
Anyone see the difference is a couple percent
And of course, I read it Cambridge Analytica... I cannot image what kind of damage that company gave to Cambridge University.
Cambridge is a city. There are many organisations calling themselves Cambridge (or Oxford for that matter), either because they are genuinely based there or because they hope to benefit from the association. Can't really trademark a city name though.
Let’s not forget that it was Cambridge who first identified the political opportunities of knowing people’s psychological tendencies through survey collection on Facebook - before Cambridge Analytica was born.
Can’t help but feel that this is slightly on the nose.
Let’s not forget that it was Cambridge who first identified the political opportunities of knowing people’s psychological tendencies through survey collection on Facebook - before Cambridge Analytica was born.
Can’t help but feel that doing something similar with cognition is slightly on the nose.