How to get R to parse the <study_design> field from XML files

by helpfully provides a facility for downloading machine-readable XML files of its data. Here’s an example of a zipped file of 10 XML files.

Unfortunately, a big zipped folder of XML files is not that helpful. Even after parsing a whole bunch of trials into a single data frame in R, there are a few fields that are written in the least useful format ever. For example, the <study_design> field usually looks something like this:

Allocation: Non-Randomized, Endpoint Classification: Safety Study, Intervention Model: Single Group Assignment, Masking: Open Label, Primary Purpose: Treatment

So, I wrote a little R script to help us all out. Do a search on, then save the unzipped search result in a new directory called search_result/ in your ~/Downloads/ folder. The following script will parse through each XML file in that directory, putting each one in a new data frame called “trials”, then it will explode the <study_design> field into individual columns.

So for example, based on the example field above, it would create new columns called “Allocation”, “Endpoint_Classification”, “Intervention_Model”, “Masking”, and “Primary_Purpose”, populated with the corresponding data.

require ("XML")
require ("plyr")

# Change path as necessary
path = "~/Downloads/search_result/"

xml_file_names <- dir(path, pattern = ".xml")

counter <- 1

# Makes data frame by looping through every XML file in the specified directory
for ( xml_file_name in xml_file_names ) {
  xmlfile <- xmlTreeParse(xml_file_name)
  xmltop <- xmlRoot(xmlfile)
  data <- xmlSApply(xmltop, function(x) xmlSApply(x, xmlValue))
  if ( counter == 1 ) {
    trials <- data.frame(t(data), row.names = NULL)
  } else {
    newrow <- data.frame(t(data), row.names = NULL)
    trials <- rbind.fill (trials, newrow)
  # This will be good for very large sets of XML files
  print (
      " processed (",
      format(100 * counter / length(xml_file_names), digits = 2),
      "% complete)"
  counter <- counter + 1

# Data frame has been constructed. Comment out the following two loops
# (until the "un-cluttering" part) in the case that you are not interested
# in exploding the <study_design> column.

columns = vector();

for ( stu_des in trials$study_design ) {
  # splits by commas NOT in parentheses
  for (pair in strsplit( stu_des, ", *(?![^()]*\\))", perl=TRUE)) {
    newcol <- substr( pair, 0, regexpr(':', pair) - 1 )
    columns <- c(columns, newcol)

for ( newcol in unique(columns) ) {
  # get rid of spaces and special characters
  newcol <- gsub('([[:punct:]])|\\s+','_', newcol)
  if (newcol != "") {
    # add the new column
    trials[,newcol] <- NA
    i <- 1
    for ( stu_des2 in trials$study_design ) {
      for (pairs in strsplit( stu_des2, ", *(?![^()]*\\))", perl=TRUE)) {
        for (pair in pairs) {
          if ( gsub('([[:punct:]])|\\s+','_', substr( pair, 0, regexpr(':', pair) - 1 )) == newcol ) {
            trials[i, ncol(trials)] <- substr( pair, regexpr(':', pair) + 2, 100000 )
      i <- i+1

# Un-clutter the working environment

remove (i)
remove (counter)
remove (data)
remove (newcol)
remove (newrow)
remove (columns)
remove (pair)
remove (pairs)
remove (stu_des)
remove (stu_des2)
remove (xml_file_name)
remove (xml_file_names)
remove (xmlfile)
remove (xmltop)

# Get nice NCT id's

get_nct_id <- function ( row_id_info ) {
  return (unlist (row_id_info) ["nct_id"])

trials$nct_id <- lapply(trials$id_info, function(x) get_nct_id (x))

# Clean up enrolment field

trials$enrollment[trials$enrollment == "NULL"] <- NA

trials$enrollment <- as.numeric(trials$enrollment)

Useful references:



    title = {How to get R to parse the <study_design> field from XML files},
    journaltitle = {The Grey Literature},
    author = {Benjamin Gregory Carlisle},
    address = {Montreal, Canada},
    date = 2016-10-6,
    url = {}


Carlisle, Benjamin Gregory. "How to get R to parse the <study_design> field from XML files" Web blog post. The Grey Literature. 06 Oct 2016. Web. 20 Feb 2017. <>


Carlisle, Benjamin Gregory. (2016, Oct 06). How to get R to parse the <study_design> field from XML files [Web log post]. Retrieved from

The answer to the question


On October 9, inspired by the STREAM research group’s Forecasting Project, I posed a question to the Internet: “Do you know how the election is going to turn out?” I tweeted it at news anchors, MP’s, celebrities, academics, friends and family alike.

I’m very happy with the response! I got 87 predictions, and only 11 of them were what I would consider “spam.” I took those responses and analysed them to see if there were any variables that predicted better success in forecasting the result of the election.

The take-home message is: No. Nobody saw it coming. The polls had the general proportion of the vote pretty much correct, but since polls do not reflect the distribution of voters in individual ridings, the final seat count was very surprising. This may even suggest that the Liberals got the impetus for a majority result from the fact that everyone expected they would only narrowly eke out a victory over the incumbent Tories.

You can view the final report in web format or download it as a PDF.


    title = {The answer to the question},
    journaltitle = {The Grey Literature},
    author = {Benjamin Gregory Carlisle},
    address = {Montreal, Canada},
    date = 2015-10-25,
    url = {}


Carlisle, Benjamin Gregory. "The answer to the question" Web blog post. The Grey Literature. 25 Oct 2015. Web. 20 Feb 2017. <>


Carlisle, Benjamin Gregory. (2015, Oct 25). The answer to the question [Web log post]. Retrieved from

Can you predict the outcome of Canada’s 42nd federal election?


The STREAM (Studies of Translation, Ethics and Medicine) research group at McGill University, of which I’m a part, has been working on a project for the last year or so in which we elicit forecasts of clinical trial results from experts in their field. We want to see how well-calibrated clinical trialists are, and to see which members of a team are better or worse at predicting trial outcomes like patient accrual, safety events and efficacy measures.

Inspired by this, I borrowed some of the code we have been using to get forecasts from clinical trial investigators, and have applied it to the case of Canada’s 42nd federal election, and now I’m asking for you to do your best to predict how many seats each party will get, and who will win in your riding.

Let’s see how well we, as a group, can predict the outcome, and see if there are regional or demographic predictors for who is better or worse at predicting election results. The more people who make predictions, the better the data set I’ll have at the end, so please submit a forecast, and ask your friends!

The link for the forecasting tool is here:

Just to make it interesting: I will personally buy a beer for the forecaster who gives me the best prediction out of them all.* :)

* If you are younger than 18 years of age, you get a fancy coffee, not a beer. No purchase necessary, only one forecast per person. Forecaster must provide email with the prediction in order for me to contact him/her. In the case of a tie, one lucky beer-receiver will be chosen randomly. Having the beer together with me is conditional on the convenience of both parties (e.g. if you live in Vancouver or something, I’ll just figure out a way to buy you a beer remotely, since I’m in Montreal). You may consult any materials, sources, polls or whatever. This is a test of your prediction ability, not memory, after all. Prediction must be submitted by midnight on October 18, 2015.


    title = {Can you predict the outcome of Canada’s 42nd federal election?},
    journaltitle = {The Grey Literature},
    author = {Benjamin Gregory Carlisle},
    address = {Montreal, Canada},
    date = 2015-10-8,
    url = {}


Carlisle, Benjamin Gregory. "Can you predict the outcome of Canada’s 42nd federal election?" Web blog post. The Grey Literature. 08 Oct 2015. Web. 20 Feb 2017. <>


Carlisle, Benjamin Gregory. (2015, Oct 08). Can you predict the outcome of Canada’s 42nd federal election? [Web log post]. Retrieved from

Things that I wish were the election issues for 2015


Whenever you hear politicians talk in the run-up to an election, everyone already knows the sorts of things they’re going to say:

“No, you would be a worse fiscal manager! The state of the economy is your fault!”

“No, truly, you would be a worse fiscal manager! The state of the economy has always been your fault!”

What they say is always so disconnected with what I feel is important, that I made a list of things that I wish we could start talking about honestly as a country. These are my “dream election issues,” and with a few exceptions, they’re not the things that politicians (of any party) really like to talk about too much. I’ve divided these things into four broad categories, and I have included a “tl;dr” at the bottom for those who don’t want to read my very wordy ramblings about Canadian politics.

Broadly speaking, I wish that we could be talking about human rights, the state of Canadian democracy, corporate influence on politics, and a guaranteed basic income.

1. Canada’s human rights record, and what can be done to help make things better for our country’s First Nations

The current government is embarrassingly bad at these sorts of issues, despite Canada having a reputation to the contrary.

The Harper Government’s reaction to the Truth and Reconciliation report, for example, was shameful. Canada is a country that literally committed a genocide against a racial group, and the minister in charge of that portfolio decided to stay sitting during a standing ovation for a call for a national inquiry on missing missing and murdered aboriginal women. The lack of movement with regard to an inquiry is inexcusable.

The government’s passage of C-51, with the help of the Liberals, has also recently been deemed a human rights problem by the UN. Naturally, the Tories have shrugged off any such criticism.

And as long as we’re talking about human rights, trans rights are human rights. Considering how smug we as Canadians like to be about LGBTQ+ stuff, (“We have gay marriage, so everything is perfect here, right!”) we’re pretty bad at actually making things better for trans people. A promising trans rights bill was killed by the senate, because … “Meh. Can’t be bothered.”

(Don’t get me wrong, gay marriage is great and all, but if there was such a thing as a Maslovian pyramid of Things that LGBTQ+ People Need In Order To Be Equal To Everyone Else, gay marriage would be pretty much at the top of it. Sure, it makes things better, but mostly for people who have got things pretty good in the first place.)

It would be healthy to finally have an honest national conversation about what kind of a place we want Canada to be. How do we want history to judge us, when they look back at the way that we have treated our First Nations? Wouldn’t it be better if we were talking about the details of an inquiry into missing and murdered aboriginal women, or even making plans to help preserve the culture of the First Nations, by making their languages official?

Or maybe we should sit down and talk about what kinds of powers we want to grant to a peacetime government. Do we want to be constantly at war with “terror,” or to have a government that demands 3 terror-related statements per week in order to keep us properly terrified? (How about this for a rule regarding the powers of the police and our country’s spies? If a police officer in 1950 would have probably needed a warrant to get access to that detail about your personal life, spies and polices officers in 2015 aren’t allowed to scoop it up in mass surveillance, or obtain that detail of your life otherwise through electronic means.)

2. Re-establishing some semblance of democracy in Canada

Over the past little while, democracy in Canada has gone downhill, fast. Before some neckbeard tries to say, “We are a democracy. We have elections, and that makes us one,” democracy is something that a country has in degrees. Just because a country has elected leaders doesn’t make it an ideal and perfect democracy. There are a number of institutions, practices, conventions and expectations that all work together to ensure that a country has a government of/by/for the people. The simple fact of an elected government, while a cornerstone of democracy, does not exhaust the meaning of the word.

This means that a number of changes to things that seem to be unrelated to whether or not Canada is a democracy can become real threats to Canada as a democracy. For example, the muzzling of Canadian scientists. If Canadian scientists can’t communicate their findings, or if only findings that can be spun to support the government of the day are released, then that eliminates a legitimate means that Canadians have to make their own opinions about how the country is run, what they communicate to their representatives, how they vote, etc.

Further, our prime minister is inaccessible to reporters and journalists. This is actually a problem for democracy itself. The media exists in part to hold the government of the day accountable, and the less accountable a government is to its people, the less democratic it is.

Same thing goes (in principle) for Question Period. The nearly comic performance of “Crocodile Tears” Calandra wasn’t just an exercise in partisan buffoonery. The whole point of parliament is that one of the mechanisms by which laws are made in our system is by talking about them. This is why omnibus bills are also fundamentally un-democratic. The process by which bills become law is supposed to be one where a government has to answer publicly for why it is making the rules that it is. An omnibus bill skips over that whole process, and is antithetical to democracy itself.

3. Corporate influence, intellectual property and protecting the public domain

As long as we’re talking about omnibus bills, let’s talk about what the Tories slipped into the last one and hoped we’d never notice. Up until recently, recordings were protected by copyright in Canada for 50 years after the death of the artist. Harper has quietly upped that figure to 70 years after the death of the artist. There was no debate on this issue. I have never heard a good justification for it. But now it’s law.

For books (at least for the time being) copyright still only extends to 50 years after the death of the writer, and after that point it falls into the Public Domain. (In the US and the UK, it’s already 70 years for books.) The fact that books eventually become Public Domain is why Project Gutenberg can exist. Project Gutenberg is an initiative to make available online—for free—books whose copyright has expired. You can download them, share them, edit them, make fanfiction, whatever. The books are free in every sense of the word.

The fact that copyright expires 20 years earlier in Canada is why Project Gutenberg Canada can exist. Authors like Ian Flemming, CS Lewis, George Orwell, etc., who died more than 50 years ago, but less than 70 years ago, can be downloaded legally for free in Canada, since there are no laws here protecting their copyright. THIS IS A GOOD THING.

There is absolutely no reason, other than corporate greed, to extend copyright after the artist has died. I mean, if copyright is protected for 70 years after the artist has died, everyone that artist ever loved or ever knew would also likely be dead before the copyright expires. The rationale, I take it, for copyright is to encourage new creative works. I am extremely sceptical that the profits of recording or publishing companies, decades after an artist’s death, is a motivating factor for any artist.

Copyright extensions are just one fairly minor issue, but they are like the canary in a coal mine. Copyright extension laws don’t try to pretend to be about the public good. They are absolutely not about the public good, and they are so transparently against the public good that they have to be snuck into law through the use of omnibus bills. The reason they are being legislated is because of corporate influence on our politics. There is no other reason. And that’s why they should be opposed—not just because I like free books (although there is that too), but because as a law that has absolutely no basis in the public good, they stand as a really simple metric for how corrupt our politics are.

4. Guaranteed basic income for all Canadians

I would love to see a guaranteed basic income as a part of a party’s election platform. To me, the phrase “make a living” is absolutely abhorrent. I just can’t justify in my mind the idea that if a human being doesn’t contribute enough to our economy, then that person literally deserves to die of exposure or starvation.

We act like poverty is an intractable problem that we can never solve. Meanwhile, the city of Medicine Hat recently eliminated homelessness by just … giving houses to those who need them. It’s supported in principle by the current mayors of Edmonton and Calgary, and by economists from all political stripes, including ones as far to the right as Milton Friedman. Basic income is not some lefty fantasy.

A few decades ago, Canada didn’t have socialised medicine, and now it’s a point of national pride that no Canadian has to stress about going broke from a healthcare emergency. I would love to see a party campaign on a promise to introduce a guaranteed basic income for Canadians, so that in a few years, it’s a point of national pride that no Canadian has to stress about going broke, ever.


The things I’d love to see become “election issues” for Canada in 2015 are the following:

  • Missing and murdered aboriginal women
  • First Nations languages as official languages of Canada
  • Trans rights
  • The powers of a peacetime government against private liberty
  • Muzzling of Canadian scientists
  • The legality of omnibus bills
  • Copyright extensions as a proxy for corporate political influence
  • Guaranteed basic income

Edit (2015 July 27): Added item #4, basic income.

Cross-posted to:


    title = {Things that I wish were the election issues for 2015},
    journaltitle = {The Grey Literature},
    author = {Benjamin Gregory Carlisle},
    address = {Montreal, Canada},
    date = 2015-07-25,
    url = {}


Carlisle, Benjamin Gregory. "Things that I wish were the election issues for 2015" Web blog post. The Grey Literature. 25 Jul 2015. Web. 20 Feb 2017. <>


Carlisle, Benjamin Gregory. (2015, Jul 25). Things that I wish were the election issues for 2015 [Web log post]. Retrieved from

Short story prompt for Lojban enthusiasts: la cizra mensi


Short story prompt: la cizra mensi

The hero of your short story has found a way to summon the Weird Sisters of Macbeth fame to inquire after the future. Worried that the witches will try to trick your hero by giving a prophesy that can be favourably and plausibly read one way, but that also has an alternate, surprising and terrible interpretation that is consistent with the words of the prophesy, your hero finds a way to force the witches to speak in Lojban.

Unfortunately for the hero of your story, a witch’s prophesy can backfire in unexpected ways that still respect the letter of the prophesy itself, even if it’s delivered in a language that’s syntactically unambiguous.

Macbeth 1.3

In the spirit of this short story prompt, I have rendered the first part of Macbeth, act 1 scene 3 into Lojban for your enjoyment. Corrections and suggestions welcome. :)

termafyfe’i 1: [1] .i doi lo mensi do pu zvati ma

termafyfe’i 2 .i lo jai bu’u lo nu catra lo xarju

termafyfe’i 3 .i doi lo mensi do zvati ma

termafyfe’i 1 .i lo fetspe be lo blopre pu cpana be lo galtupcra ku ralte lo narge

[5] gi’e omnomo gi’e omnomo gi’e omnomo .i lu ko dunda fi mi li’u se cusku mi .i lu ko cliva doi lo termafyfe’i li’u lo zargu citka cagna cu se krixa .i lo nakspe be lo se go’i pu klama la .alepos. gi’e bloja’a la .tirxu. .i ku’i ne’i lo julne mi lo te go’i fankla

[10] .ije mi simsa be lo ratcu poi claxu lo rebla ku co’e gi’e co’e gi’e co’e

termafyfe’i 2: .i mi dunda do pa lo brife

termafyfe’i 1 .i do xendo

termafyfe’i 3 .i mi co’e pa lo drata

termafyfe’i 1: [15] .i mi ralte ro da poi drata .i je’a lo blotcana cu bifca’e ro da poi farna be fi lo makfartci pe lo blopre ku’o zi’e poi se djuno .i mi ba simsa be lo sudysrasu bei lo ka sudga ku rincygau

[20] .i lo nu sipna ku ba canai lo donri ku .a lo nicte ku dandu za’e lo galtu dinju canko gacri .i zo’e ba dapma renvi .i ba ca lo tatpi jeftu be li so pi’i so cu jdika lo ka stali .e lo ka pacna .e lo ka gleki

[25] .i zu’u lo bloti to’e pu’i se daspo .i zu’unai lo go’i vilti’a se renro .i ko viska lo se ralte be mi

termafyfe’i 2: .i ko jarco fi mi .i ko jarco fi mi

termafyfe’i 1 .i mi nau ralte lo tamji be fi lo blosazri

[30] poi ca lo nu zdani klama ku bloti janli morsi

[.i ne’i damri]

termafyfe’i 3: .i damri .i damri .ua .i la .makbet. je’a tolcliva

ro da poi termafyfe’i: .i lo cizra mensi noi xance jgari simxu zi’e noi klama be fo lo xamsi .e lo tumla be’o sutra

[35] cu klama fi’o tadji tu’a di’e .i ciroi klama lo tu’a do .i ciroi klama lo tu’a mi .i ciroi ji’a klama .iki’ubo krefu fi li so .i ko smaji .i lo makfa cu bredi

[.i nerkla fa la .makbet. .e la bankos.]


    title = {Short story prompt for Lojban enthusiasts: la cizra mensi},
    journaltitle = {The Grey Literature},
    author = {Benjamin Gregory Carlisle},
    address = {Montreal, Canada},
    date = 2015-07-1,
    url = {}


Carlisle, Benjamin Gregory. "Short story prompt for Lojban enthusiasts: la cizra mensi" Web blog post. The Grey Literature. 01 Jul 2015. Web. 20 Feb 2017. <>


Carlisle, Benjamin Gregory. (2015, Jul 01). Short story prompt for Lojban enthusiasts: la cizra mensi [Web log post]. Retrieved from

Time to set your clocks forward and feel terrible again


Everyone’s gonna write an article about how we should drop Daylight Savings Time. That’s all well and good, but I say it doesn’t go far enough.

The 12 hour clock has got to go

We should outlaw the 12-hour clock. Why on earth do we have a convention that has a really obvious ambiguity built into it?

“We’re meeting at eight.” What does that mean? Without clarification, you don’t know. It’s not that hard. Make the switch today. There’s no reason to re-use numbers. It’s not like we’re going to run out of them.

It’ll take you about 2 days before you’re used to it.

Time zones too, then!

We should also eliminate time zones. Have you ever tried setting up a telephone meeting with someone who lives in another time zone? It’s terrible, and it’s even worse when it’s around the time that DST ends or begins. And it’s not like the sun is directly overhead at 12:00 with the system we’ve got anyway.

Being able to instantly know that 9h00 means “mid-morning” regardless of the geographic location of the speaker isn’t that big of a thing. In fact, we have another way to express that idea: by saying “mid-morning.” We should use numbers to designate times when we want exactness and the ability to coordinate. We should use short phrases when we want to give general impressions regarding the position of the sun in the sky relative to the speaker. Trying to get our numbers to do both makes them less able to do either, and I say we should let numeric times do what they’re meant to, namely, helping us to coordinate ourselves with other humans around the world.

So everyone, reset your watch to UTC. For those of us in Montreal, that means go 5 hours ahead. From now on, a typical workday in Montreal will start at 14h00 and end at 22h00, which might look confusing, but then most of you were using the 12-hour clock and so you didn’t have associations with those numbers anyway. You can get used to it at the same time that you’re getting used to not using the 12-hour clock.

More you say?

Now that you mention it, 24 hours in a day is a weird number of hours. Let’s round that up to a good solid 25. We’ll call them “metric hours.” If we divide each day into 25 metric hours, each one would be 57 m 36 s long. That’s great! No one will even notice the missing 2 m 24 s we took from each hour.

Also, a metric decihour (5 m 45.6 s) makes for a good, natural division of time, as does the centihour (34.56 s). We just need to get used to saying things like, “Your presentation must be between 1-2 decihours long,” or “Steep your tea for 6 centihours.”

Even better, we can do away with the 7-day week in favour of the 4-day hectohour (25 hours x 4 days = 1 hectohour). That way each week has a nice round 100 metric hours in it. Of course, we would have to alternate between 1- and 2-day weekends, in order to ensure that we don’t lose any weekend time, but that would mean a nearly 10% increase in weekend time. Perfect!

To wrap up

  • Daylight Savings Time is evil
  • 12-hour clocks are stupid
  • Time zones cause so many problems
  • Screw it, nobody actually wants the system to make sense


    title = {Time to set your clocks forward and feel terrible again},
    journaltitle = {The Grey Literature},
    author = {Benjamin Gregory Carlisle},
    address = {Montreal, Canada},
    date = 2015-03-6,
    url = {}


Carlisle, Benjamin Gregory. "Time to set your clocks forward and feel terrible again" Web blog post. The Grey Literature. 06 Mar 2015. Web. 20 Feb 2017. <>


Carlisle, Benjamin Gregory. (2015, Mar 06). Time to set your clocks forward and feel terrible again [Web log post]. Retrieved from

I’m just gonna throw it out there—vaccine denialism technically fits the Canadian Criminal Code’s definition for “terrorist activity”


I’m going to preface this by saying that I’m not a lawyer, and this post is meant more for comic effect than to be an actual attempt at a serious legal analysis. With that caveat firmly in place, maybe we can use some of these terrible new anti-terrorism powers that the government has given itself to combat vaccine denialism?

Section Quote from Criminal Code Does it apply to vaccine denialists?
83.01 (1) “Terrorist activity” means …
(b) … an act or omission, in or outside Canada … Failing to vaccinate one’s child, or advocating against the vaccination of children counts as an act or omission committed in or outside Canada.
(i) (A) … that is committed in whole or in part for a political, religious or ideological purpose, objective or cause, and … This definitely counts as “political” and “ideological.”
(i) (B) … in whole or in part with the intention of intimidating the public, or a segment of the public, with regard to its security, including its economic security, or compelling a person, a government or a domestic or an international organization to do or to refrain from doing any act, whether the public or the person, government or organization is inside or outside Canada, and … While the intention is not to intimidate anyone with regard to their security, advocating for a position that denies the clear consensus of the scientific community with regard to vaccine safety certainly does have the intention of compelling a person to refrain from an act, namely, vaccinating a child, which meets this definition. There are also other actors (doctors, teachers, etc.) who are also targets of this movement.
(ii) (C) … that intentionally … causes a serious risk to the health or safety of the public or any segment of the public Vaccine denialism does cause a serious risk to the health and safety of the public.
Source: Criminal Code, RSC 1985, c C-46


    title = {I’m just gonna throw it out there—vaccine denialism technically fits the Canadian Criminal Code’s definition for “terrorist activity”},
    journaltitle = {The Grey Literature},
    author = {Benjamin Gregory Carlisle},
    address = {Montreal, Canada},
    date = 2015-02-6,
    url = {}


Carlisle, Benjamin Gregory. "I’m just gonna throw it out there—vaccine denialism technically fits the Canadian Criminal Code’s definition for “terrorist activity”" Web blog post. The Grey Literature. 06 Feb 2015. Web. 20 Feb 2017. <>


Carlisle, Benjamin Gregory. (2015, Feb 06). I’m just gonna throw it out there—vaccine denialism technically fits the Canadian Criminal Code’s definition for “terrorist activity” [Web log post]. Retrieved from

Switching to left-handed Dvorak


I’m doing an experiment. A lot of my thesis work consists of me clicking between form elements, spreadsheet cells, or parts of text documents, entering short bits of text and then clicking away to another thing.

I’ve been trying to bring my efficiency up, but running into a wall. The rate-limiting step in my workflow is not my typing speed or how quickly I find information, but rather how fast I can switch from mouse to keyboard.

A few years back, I switched from QWERTY to Dvorak, which was distressing at the time, but turned out to have been an excellent life choice. (Highly recommended!) I’m going to try left-handed Dvorak out for a bit and see how it takes. :)


    title = {Switching to left-handed Dvorak},
    journaltitle = {The Grey Literature},
    author = {Benjamin Gregory Carlisle},
    address = {Montreal, Canada},
    date = 2015-01-14,
    url = {}


Carlisle, Benjamin Gregory. "Switching to left-handed Dvorak" Web blog post. The Grey Literature. 14 Jan 2015. Web. 20 Feb 2017. <>


Carlisle, Benjamin Gregory. (2015, Jan 14). Switching to left-handed Dvorak [Web log post]. Retrieved from

Dr Susan’s counselling service for para-magical, epi-paranormal and time-travel adjacent children and young adults


Dr Susan's Counselling Service

Dr Susan’s Counselling Service

For this year’s upcoming NaNoWriMo, I think I have settled on an idea and a title.

A recurring trope in sci-fi and fantasy is the minor character who significantly helps the main character to accomplish a fantastical and difficult-to-believe goal (e.g. returning to her own non-dystopian timeline, saving a magical kingdom, etc.), and does so often at great cost to herself, without any hope of participating in that victory, and with little or no proof that anything of importance happened at all. I want to write a series of short stories about this sort of character, and a therapist whose job it is to help them pick up the pieces of their shattered lives, after they discover that they’re living in a dystopian timeline bad enough that a time-traveller needed their help to go back in time to prevent it, leaving them hopelessly behind.

The title will be Dr Susan’s counselling service for para-magical, epi-paranormal and time-travel adjacent children and young adults.


    title = {Dr Susan’s counselling service for para-magical, epi-paranormal and time-travel adjacent children and young adults},
    journaltitle = {The Grey Literature},
    author = {Benjamin Gregory Carlisle},
    address = {Montreal, Canada},
    date = 2014-09-14,
    url = {}


Carlisle, Benjamin Gregory. "Dr Susan’s counselling service for para-magical, epi-paranormal and time-travel adjacent children and young adults" Web blog post. The Grey Literature. 14 Sep 2014. Web. 20 Feb 2017. <>


Carlisle, Benjamin Gregory. (2014, Sep 14). Dr Susan’s counselling service for para-magical, epi-paranormal and time-travel adjacent children and young adults [Web log post]. Retrieved from

Proof of prespecified endpoints in medical research with the bitcoin blockchain



The gerrymandering of endpoints or analytic strategies in medical research is a serious ethical issue. “Fishing expeditions” for statistically significant relationships among trial data or meta-analytic samples can confound proper inference by statistical multiplicity. This may undermine the validity of research findings, and even threaten a favourable balance of patient risk and benefit in certain clinical trials. “Changing the goalposts” for a clinical trial or a meta-analysis when a desired endpoint is not reached is another troubling example of a potential scientific fraud that is possible when endpoints are not specified in advance.

Pre-specifying endpoints

Choosing endpoints to be measured and analyses to be performed in advance of conducting a study is a hallmark of good research practice. However, if a protocol is published on an author’s own web site, it is trivial for an author to retroactively alter her own “pre-specified” goals to align with the objectives pursued in the final publication. Even a researcher who is acting in good faith may find it less than compelling to tell her readers that endpoints were pre-specified, with only her word as a guarantee.

Advising a researcher to publish her protocol in an independent venue such as a journal or a clinical trial registry in advance of conducting research does not solve this problem, and even creates some new ones. Publishing a methods paper is a lengthy and costly process with no guarantee of success—it may not be possible to find a journal interested in publishing your protocol.

Pre-specifying endpoints in a clinical trial registry may be feasible for clinical trials, but these registries are not open to meta-analytic projects. Further, clinical trial registry entries may be changed, and it is much more difficult (although still possible) to download previous versions of trial registries than it is to retrieve the current one. For example, there is still no way to automate downloading of XML-formatted historical trial data from in the same way that the current version of trial data can be automatically downloaded and processed. Burying clinical trial data in the “history” of a registry is not a difficult task.

Publishing analyses to be performed prior to executing the research itself potentially sets up a researcher to have her project “scooped” by a faster or better-funded rival research group who finds her question interesting.

Using the bitcoin blockchain to prove a document’s existence at a certain time

Bitcoin uses a distributed, permanent, timestamped, public ledger of all transactions (called a “blockchain”) to establish which addresses have been credited with how many bitcoins. The blockchain indirectly provides a method for establishing the existence of a document at particular time that can be independently verified by any interested party, without relying on a medical researcher’s moral character or the authority (or longevity) of a central registry. Even in the case that the NIH’s servers were destroyed by a natural disaster, if there were any full bitcoin nodes left running in the world, the method described below could be used to confirm that a paper’s analytic method was established at the time the authors claim.


  1. Prepare a document containing the protocol, including explicitly pre-specified endpoints and all prospectively planned analyses. I recommend using a non-proprietary document format (e.g. an unformatted text file or a LaTeX source file).
  2. Calculate the document’s SHA256 digest and convert it to a bitcoin private key.
  3. Import this private key into a bitcoin wallet, and send an arbitrary amount of bitcoin to its corresponding public address. After the transaction is complete, I recommend emptying the bitcoin from that address to another address that only you control, as anyone given the document prepared in (1) will have the ability to generate the private key and spend the funds you just sent to it.


The incorporation into the blockchain of the first transaction using the address generated from the SHA256 digest of the document provides an undeniably timestamped record that the research protocol prepared in (1) is at least as old as the transaction in question. Care must be taken not to accidentally modify the protocol after this point, since only an exact copy of the original protocol will generate an identical SHA256 digest. Even the alteration of a single character will make the document fail an authentication test.

To prove a document’s existence at a certain point in time, a researcher need only provide the document in question. Any computer would be able to calculate its SHA256 digest and convert to a private key with its corresponding public address. Anyone can search for transactions on the blockchain that involve this address, and check the date when the transaction happened, proving that the document must have existed at least as early as that date.


This strategy would prevent a researcher from retroactively changing an endpoint or adding / excluding analyses after seeing the results of her study. It is simple, economical, trustless, non-proprietary, independently verifiable, and provides no opportunity for other researchers to steal the methods or goals of a project before its completion.

Unfortunately, this method would not prevent a malicious team of researchers from preparing multiple such documents in advance, in anticipation of a need to defraud the medical research establishment. To be clear, under a system as described above, retroactively changing endpoints would no longer be a question of simply deleting a paragraph in a Word document or in a trial registry. This level of dishonesty would require planning in advance (in some cases months or years), detailed anticipation of multiple contingencies, and in many cases, the cooperation of multiple members of a research team. At that point, it would probably be easier to just fake the numbers than it would be to have a folder full of blockchain-timestamped protocols with different endpoints, ready in case the endpoints need to be changed.

Further, keeping a folder of blockchain-timestamped protocols would be a very risky pursuit—all it would take is a single honest researcher in the lab to find those protocols, and she would have a permanent, undeniable and independently verifiable proof of the scientific fraud.


Fraud in scientific methods erodes confidence in the medical research establishment, which is essential to it performing its function—generating new scientific knowledge, and cases where pre-specified endpoints are retroactively changed casts doubt on the rest of medical research. A method by which anyone can verify the existence of a particular detailed protocol prior to research would lend support to the credibility of medical research, and be one less thing about which researchers have to say, “trust me.”


    title = {Proof of prespecified endpoints in medical research with the bitcoin blockchain},
    journaltitle = {The Grey Literature},
    author = {Benjamin Gregory Carlisle},
    address = {Montreal, Canada},
    date = 2014-08-25,
    url = {}


Carlisle, Benjamin Gregory. "Proof of prespecified endpoints in medical research with the bitcoin blockchain" Web blog post. The Grey Literature. 25 Aug 2014. Web. 20 Feb 2017. <>


Carlisle, Benjamin Gregory. (2014, Aug 25). Proof of prespecified endpoints in medical research with the bitcoin blockchain [Web log post]. Retrieved from


A word from our sponsors

Tag bag

Recent comments

Old posts

All content © Benjamin Carlisle