High-resolution Risk of Bias assessment graph… in Excel!

Some years ago, I found myself ranting and raving at the RevMan software kit, which is the official Cochrane Collaboration software suite for doing systematic reviews. Unfortunately, either because I’m an idiot or because the software is an idiot (possibly both), I found it impossible to export a Risk of Bias assessment graph at a resolution that was even remotely acceptable to journals. These days journals tend to accept only vector-based graphics or bitmap images in HUGE resolutions (presumably so they can scale these down to unreadable smudges embedded in a .pdf). At that time I had a number of meta-analyses on my hands so I decided to recreate the RevMan-style risk of bias assessment graph, but in Excel. This way anyone can make crisp-looking risk of bias assessment graphs at a resolution higher than 16dpi (or whatever pre-1990 graphics resolution RevMan appears to use…)

The sheet is relatively easy to use, just follow the embedded instructions. You need (1) percentages from your own risk of bias assessment (2) basic colouring skills that I’m sure you’ve picked up before the age of 3. All you basically do to make the risk of bias assessment graph is colour it in using Excel. It does involve a bit of fiddling with column and row heights and widths, but it gives you nice graphs like these:

Sample Risk of Risk of Bias assessment graph

Sample Risk of Bias Graph

Like anything I ever do, this comes with absolutely no guarantee of any kind, so don’t blame me if this Excel file blows up your computer, kills your pets, unleashes the Zombie Apocalypse or makes Jason Donovan record a new album.


 

Download available here (licensed under Creative Commons BY-SA):

Risk of Bias Graph in Excel – v2.3

MD5 checksum: 9D598B77B9151A48D803FA08817EA0C3


 

 

eMental Health interview with VGCt [Dutch]

Nothing like an interview on eMental Health to make you feel important

I’m still reeling from the festivities surrounding my H-index increase from 3 (“aggressively mediocre“) to 4 (“impressively flaccid but with mounting tumescence“)*. Best gift I got: a sad, weary stare from my colleagues. Yay! But back to eMental Health (booooo hisssss).

Some while back I did an interview (in Dutch) with Anja Greeven from the Dutch Association for Cognitive Behavioural Therapy [Vereniging voor Gedragstherapie en Cognitieve Therapie] for their Science Update newsletter in December 2015. It’s about life, the universe and everything; but mostly about eHealth and eMental Health; implementation (or lack thereof), wishful thinking, perverse incentives (you have a filthy mind) and that robot therapist we’ve all been dreaming about (sorry, Alan Turing).

Kudos to me for the wonderful contradiction where I call everyone predicting the future a liar and a charlatan; after which I blithely shoot myself in the foot by trying to predict the future. In my defense, I never claimed I wasn’t a liar and a charlatan. It was great fun blathering on about all kinds of things, and massive respect to Anja who had to wade through a 2-hour recording of my irritating voice to find things that might pass as making sense to someone, presumably.

Anyway, the interview is in Dutch, so good luck Google Translating it!


Link to the VGCt interview in .pdf [Dutch]

 

*) Real proper technical sciencey descriptions for these numbers, actually. The views expressed in this interview are my own; and nobody I know or work for would ever endorse the silly incoherent drivel I’ve put forward in this interview.

Save the Data! Data integrity in Academia

Data integrity is integral to reproducibility.

I recently read something on an Internet web site called Facebook, it’s supposed to be quite the thing at the moment. Friend and skeptical academic James Coyne, whose fearless stabs at the methodologically pathetic and conceptually weak I much admire, instafacetweetbooked a post over at Mind the Brain, pointing to a case in post-publication peer review that made me wonder whether I was looking at serious academic discourse or toddlers in kindergarten trying to smash each other’s sand castles. James and I have co-authored a manuscript about the shortcomings in psychotherapy research which is available freely here, and I’m ashamed so say that I still haven’t met up with James in person, although he’s tried to get a hold of me more than once when he was in Amsterdam.

Anyway, point in case, during post-publication peer review where these reviewers highlighted flaws in the original analysis, the original authors had manipulated the published data to pretend the post-publication peer reviewers were a bunch of idiots who didn’t know what they’re doing. This is clearly pathetic and must have been immensely frustrating for the post-publication reviewers (it was a heroic feat in itself to be able to prove such devious manipulations in the first place, thankfully they took close note of the data set time stamps).

What can be done? Checking time stamps is trivial, but so is manipulating time stamps. My mind immediate took to what nerdy computery types like me have used for a very, very long time: file checksums. We use these things to check whether, for example, the file we just downloaded didn’t get corrupted somewhere along the sewer pipes of the Internet. Best known, probably, are MD5-hashes, a cryptographic hash of the information in a file. MD5-hases are unique: they are composed of 32 alphanumeric characters (A-Z, 0-9) which yields (26+10)^32 = 6,3340286662973277706162286946812e+49 different combinations. That’ll do nicely to catalogue all the Internet’s cat memes with unique hashes from decades past and aeons to come, and then some. So, if I were to download nyancat.png from www.nyancatmemerepository.com, I could calculate the hash of that downloaded file using, e.g., the excellent md5check.exe by Angus Johnson, which gives me a unique 32-character hash; which I could then compare with the hash as shown on www.nyancatmemerepository.com. Few things are worse than corrupted cat memes, really, but let’s consider that these hashes are equally useful to check whether a piece of, say, security software, wasn’t tampered with somewhere between the programmer’s keyboard and your hard drive – it’s the computer equivalent of putting a tiny sliver of sellotape on the cookie jar to see that nobody’s nicking your Oreos.

How can all this help us in science and the case stated above? Let’s try to corrupt some data. Let’s look at the SPSS sample data file “anticonvulsants.sav” as included in IBM SPSS21. It’s a straightforward data set looking at a multi-centre pharmacological intervention for an anticonvulsant vs. placebo, for patients followed for a number of weeks, reporting number of convulsions per patient per week as a continuous scale variable. The MD5 hash (“checksum”) for this data file is F5942356205BF75AD7EDFF103BABC6D3 as reported by md5check.exe.

screenie1

First, I duplicate the file (anticonvulsants (2).sav), and md5check.exe tells me that the checksum matches with the original [screenshot] – these files are bit-for-bit exactly the same. The more astute observer will wonder why changing the filename didn’t change the checksum (bit-for-bit, right?). Let’s not go into that much detail, but Google most assuredly is your friend if you really must know.

Now, to test the anti-tamper check, let’s say we’re being mildly optimistic about the number of convulsions that our new anticonvulsant can prevent. Let’s look at patient 1FSL from centre 07057.  He’s on our swanky new anticonvulsant, and the variable ‘convulsions’ tells us he’s had 2, 6, 4, 4, 6 and 3 convulsions each week, respectively. But I’m sure the nurses didn’t mean to report that. Perhaps they mistook his spasmodic exuberance during spongey-bathtime as a convulsion? Anyway. I’m sure they meant to report 2 fewer convulsions per week as he gets the sponge twice a week, so I subtract 2 convulsions for each week, leaving us with 0, 4, 2, 2, 4 and 1 convulsions.

Let’s save the file and, compare checksums against the original data file.

screenie2

Oh dear. The data done broke. The resulting checksum for the… enhanced dataset is E3A79623A681AD7C9CD7AE6181806E8A, which is completely different from the original checksum, which was F5942356205BF75AD7EDFF103BABC6D3 (are you convulsing yet?).

Since the MD5-hashes are unique, changing just a single bit of information in a data file compromises data integrity; and regular numbers take up more than just one bit of information. Be it data corruption or malicious intent, if there’s a mismatch in files then there’s a problem. Is this a good point to remind you that replication is a fundamental underpinning of science? Yes it is.

This was just a simple proof-of-concept and I sure this has been done before. The wealth of ‘open data’ means that data are – to both honest re-analysis and dishonest re-analysis. To ensure data-integrity, when graciously uploading raw data with a manuscript, why not include some kind of digital watermark? In this example, I’ve used the humble (and quite vulnerable) MD5-hash to show how a untampered dataset would pass the checksum test, making sure that re-analysts are all singing from the same datasheet as the original authors, to horribly butcher a metaphor. Might I suggest, “Supplement A1. Raw Data File. MD5 checksum F5942356205BF75AD7EDFF103BABC6D3”.

 

H-index update: still pathetic.

Oh lookie – my H-index went up.

That is academic-speak for “my dick just got a bit less small“. The H-index rose from 2 (“oppressively pathetic“) to 3 (“aggressively mediocre“). At some point this year maybe it’ll rise to 4 – “impressively flaccid but with mounting tumescence“.

For an explanation of what all the hubbub is about, check out the Wikipedia page on the H-index. TL;DR: I have 3 papers which have been cited at least three times. My best-cited paper is still the systematic review “Persuasive System Design Does Matter: A systematic review of adherence to web-based interventions” at JMIR (I’m second author).

 

-RK

New paper in the bulletin of the EHPS

765-823-1-PB-1

What’s up with the speed of eHealth implementation?

Fresh off the virtual press at the bulletin of the European Health Psychology Society: Jeroen Ruwaard and I investigate into the rapid pace of eHealth implementation. Many bemoan the slow implementation and uptake of eHealth, but aren’t we in fact going too quickly? We examine four arguments to implement unvalidated (i.e., not evidence-based) interventions and find them quite lacking in substance, if not style.

Ruwaard, J. J., & Kok, R. N. (2015). Wild West eHealth: Time to Hold our Horses? The European Health Psychologist, 17(1).

Download the fulltext here [free, licensed under CC-BY].

Re-amping Ritual, Rejoice!

In preparing the re-issue of our critically acclaimed but sold out 2003 debut album The Apotheosis, we decided to have a little fun and re-record a few of the old tracks, just a tad shy of 10 years later. And wow, has technology come a long way since 2002/2003. We now do basically everything ourselves. No wait, we literally do everything ourselves, apart from mixing and mastering. Most of us still remember fiddling about on little 4-track Tascam recorders that used ordinary cassette tapes, nowadays we do 8-track digital stuff in Protools in unimaginable sound quality without even batting an eyelid. Now, I’ve always been a big fan of re-amping.

The contenders. Left to right: Røde NT1000, Shure SM58, Audio Technica ATM25 (x2), Audio Technica ATM21, Audio Technica ATM31R, Audio Technica AT4033a, AKG D112.

The contenders. Left to right: Røde NT1000, Shure SM58, Audio Technica ATM25 (x2), Audio Technica ATM21, Audio Technica ATM31R, Audio Technica AT4033a, AKG D112.

Long story short, it means not recording a thundering amp while you play, but record just the instrument and play it back through an amp later. This has a number of advantages, but for DIY-types like us the biggest is having total control over your sound while you’re not playing. Essentially you get to be the bass player and sound engineer in one and you don’t have to play something, listen back, put down your bass, fiddle with your amp/microphone, put on your bass, play something, do it all over again, ad nauseam. Armed with a nice selection of microphones we set to with an Ampeg 8×10 loaned to us by Tom of the almighty Dead Head. I used my SVP-PRO (we are inseparable) and trusty Peavey power amp, and started experimenting with microphone placement and combinations.

D112, ATM25 and AT4033a in action. NT1000 to the far left in the corner, not in the pic.

D112, ATM25 and AT4033a in action. NT1000 to the far left in the corner, not in the pic.

The winning combination turned out to be the ATM25 off-axis, right on the edge of the cone at 45 degrees, edged back just about an inch, with the AT4033a at 70 cms (2.3 feet), just about in the vertical centre of the 8×10.

Tadaa.

Tadaa.

I used Audacity to make these cool plots, and the graphs clearly show the differences in microphone signals. At the end of the day, the D112 was too boomy anywhere near the speaker cone (the very proximity effect the D112 is ‘famous’ for), the ATM25 sounded simply more gritty, dark and… well, evil. The AT4033a complemented the ATM25 perfectly, topping off the ATM25’s low-end gurgle with a snappy, gnarly high-mid end. Interestingly, the Røde NT1000 stashed away in the far corned picked up quite some lows and mids as you can see by the huge hump below 100Hz, but I’m not sure we’re going to use it (there is quite an audible rattle in there somewhere from something vibrating).

layered-mics

Shoddily pasted graph showing the frequency responses of the different mikes in their different settings. Note the huge low-end response on the NT1000 condenser!

Here are some sound samples, straight from the board with just a touch of compression (1:2.5, 0.1msec attack, 2sec decay).

Røde NT-1000

Audio Technica ATM4033a

Audio Technica ATM25

AKG D112

Quick thought

Meta-analyses are ventriloquist’s dummies. Sitting on a wise man’s knee they may be made to utter words of wisdom; elsewhere, they say nothing, or talk nonsense, or indulge in sheer diabolism.” – Adapted from Aldous Huxley

Corrected JMIR citation style for Mendeley desktop

Endnooooooooo!te.

100 out of 100 academics agree that working with Endnote is about as enjoyable as putting your genitals through a rusty meat grinder while listening to Justin Bieber’s greatest hits at full blast and being waterboarded with liquid pig shit. I’ve spent countless hours trying to salvage the broken mess that Endnote leaves and have even lost thousands of carefully cleaned and de-duplicated references for a systematic review due to a completely moronic ‘database corruption’ that was unrecoverable.

Thankfully, there is an excellent alternative in the free, open source (FOSS) form of Mendeley Desktop, available for Windows, OS X, iToys and even Linux (yay!).

One of the big advantages of Mendeley over Endnote, apart from it not looking like the interface from a 1980s fax machine, is the ability to add, customise and share your own citation styles in the .csl (basically xml/Zotero) markup. While finishing my last revised paper I found out that the shared .csl file for the Journal of Medical Internet Research (a staple journal for my niche) is quite off and throws random, unnecessary fields in the bibliography that did not conform to JMIR’s instructions for authors.

The online repository of Mendeley is pretty wonky and the visual editor isn’t too user friendly, so I busted out some seriously nerdy h4xx0rz-skillz (which chiefly involved pressing backspace a lot) .

Get it.

Well, with some judicious hacking, I present to you a fixed JMIR .csl file for Mendeley (and probably Zotero, too). Download the JMIR .csl HERE (probably need to click ‘save as’, as your browser will try to display the xml stream). It’s got more than a few rough edges but it works for the moment. Maybe I’ll update it some time.

According to the original file, credits mostly go out to Michael Berkowitz, Sebastian Karcher and Matt Tracy. And a bit of me. And a bit of being licensed under a Creative Commons Attribution-ShareAlike 3.0 License. Don’t forget to set the Journal Abbreviation Style correctly in the Mendeley user interface.

Oh, I also have a Mendeley profile. Which may or may not be interesting. I’ve never looked at it. Tell me if there’s anything interesting there. So, TL;DR: Mendeley is FOSS (Free Open Source Software), Endnote is POSS (Piece of Shit Software).

Update: A friendly blogger from Zoteromusings informed me in the comments that I was wrong: Mendeley is indeed not FOSS but just free to use, and not open source. Endnote is still a piece of shit, though. I was right about that 😉

Academic journal recognition, not of a very savoury kind

You know an academic field has come to full maturity when the null results come rolling in. But you really know an academic discipline has come to maturity when the first shady academic journals pop up. Enter “E-Health Telecommunication Systems and Networks” from Scientific Research Publishing (love their description “An Academic Publisher“!).

Much publish. Very journal. So impact factor. Wow

An invitation mail mass-spam just popped into my spam e-mail box alongside invitations to submit to other “rapid peer review and publishing” journals with names that range from could-be reputable (‘British Journal of Medicine and Medical Research’) to downright weird. The basic premise of these journal is well-known to me as a musician: the much-maligned pay-to-play‘ principle has many a musician grunting (ha!) and complaining – it looks like the pay-to-publish racket is not very different.

Should you send an eHealth-related manuscript to these people? Is Scientific Research Publishing a predatory journal? Who knows. But the word on the virtual streets is not good. And a company publishing 200+ English-language journals in just over 6 years is ambitious, to say the least. Oh yes, and they gladly accepted a randomly generated maths paper (read: total bullshit) for publication after a mere 10 days of review – only the lead ‘author’ didn’t pony up the processing charges.

Long story short, if you want to publish your eHealth-related manuscript in one of my 666+ Bottom Of The Barrel Research Publishing Co. journals, just send me the manuscript and $2000. I’ll personally do some ‘peer-reviewing’. Or do it the boring old way and send your eHealth-related manuscript to reputable academic journals like the Journal of Medical Internet Research, Internet Interventions or the Journal of Telemedicine and Telecare.