Semantic video indexing app

by

The newest version of Mac OS, called “Mountain Lion,” includes “Dictation,” which is a piece of system software that takes speech and converts it to text. This is nothing new, of course. I remember that I had a piece of dictation software for my old Windows 98 PC. You had to “train” the software to understand what you said, and even then it was wildly inaccurate, but in principle, this sort of software has existed for a long time. Dictation on Mac OS is much better than the one I had back in 1998, but of course it is not perfect.

That particular piece of software I had on my PC was not built in to the operating system. I had to pay for it. Not only that, but because it didn’t work very well, I never got another dictation programme again. But now that this one is built into the OS, I think I’m going to try an experiment.

Here’s my inspiration: In Star Trek, every character keeps a “log,” and because it’s the future, it’s an audio log. In The Next Generation, they were often shown as video (b)logs. Sometimes, in order to advance the plot, a character would be shown searching through his own (or another person’s) logs. What was interesting was that the search would usually be a semantic keyword search. Something like, “Computer, show me all log entries relating to the warp core” (or whatever they were interested in at the time). With dictation software now a standard feature in OS X, we’re at a point where we could write an app that does exactly what the computer did in Star Trek.

The workflow will be as follows: Take a video (or a set of videos) that you’re interested in, and extract the audio. Divide the one big audio file into hundreds of smaller (say, ten-second-long), overlapping audio files that are annotated with their start time in the original video. For each of these smaller files, pass them through the dictation software and generate a text file that includes the text that has been generated by the system’s text-to-speech dictation software. And voilà, you have generated a time-encoded text index for your video—just like the one on YouTube, but you wouldn’t have to upload the file.

Wrap this all up in a shiny OS X app wrapping and put it on the App Store. Sell it for $0.99.

Then, if you had a bunch of videos—say, seasons 5–6 of Doctor Who, and you wanted to find all references to “the Silence,” you could install the app, have it index your iTunes library, and then do a search through your videos for certain keywords or phrases.

Actually, this might work. If anyone wants to collaborate with me on this one, hit me up in the comments.

Edit: I take it back. A quick experiment with Dictation indicates that we are nowhere near having the technology to be able to do this.

BibTeX

@online{bgcarlisle2012-2988,
    title = {Semantic video indexing app},
    journaltitle = {The Grey Literature},
    author = {Benjamin Gregory Carlisle},
    address = {Montreal, Canada},
    date = 2012-08-16,
    url = {https://www.bgcarlisle.com/blog/2012/08/16/semantic-video-indexing-app/}
}

MLA

Carlisle, Benjamin Gregory. "Semantic video indexing app" Web blog post. The Grey Literature. 16 Aug 2012. Web. 10 Dec 2018. <https://www.bgcarlisle.com/blog/2012/08/16/semantic-video-indexing-app/>

APA

Carlisle, Benjamin Gregory. (2012, Aug 16). Semantic video indexing app [Web log post]. Retrieved from https://www.bgcarlisle.com/blog/2012/08/16/semantic-video-indexing-app/

Human research ethics and smartphone application development

by

I’m entering the final stages of working on my thesis, and not a moment too soon—I looked up my submission deadlines, and I need to make initial submission by June 13th according to the graduate studies website.

As many of you know, I’ve recently taken up iPhone application development. Less well-known is the fact that this was partly inspired by my work on my thesis. This may seem strange due to the fact that I’m working on human research ethics, and not on computer science, but stay with me.

Phase IV drug research

My thesis has to do with phase IV drug trials. “Phase IV” means that the drug has already been approved for regular medical use in humans. The problem with phase IV trials is that they are often initiated by the marketing divisions of pharmaceutical companies for the purpose of getting physicians used to prescribing the drug in question. They often have a very inefficient study design and they do not ask or answer a valuable scientific question.

Research ethics boards have a hard time criticising these studies, and often their members have to “hold their noses” and approve such studies because there would be no risk to the subjects of such research. (Of course not—the drug is being prescribed by physicians for regular medical use already!)

My thesis argues that we should criticise such studies on the basis that they threaten the integrity of the human research project. That is to say, human research is a collaborative project that requires a number of stakeholders—the drug company, physicians, human subjects, nurses, administrators, universities, research centres, hospitals, etc. All of these stakeholders come to the human research enterprise to pursue their different goals and desires, and bring different contributions to the project of human research. Because of the way that human research is set up, there are certain rules or practices that make the project of human research work better and certain rules and practices that interfere with its proper functioning.

I argue in my thesis that we should be able to criticise certain “bad” phase IV drug studies on the basis that they compromise the integrity of human research, completely aside from any paternalistic appeal to the risks or benefits that may accrue to the participants.

How is this related to iPhone app development?

Smartphone software development is also a collaborative enterprise. That is to say, there are also many parties in smartphone software development that come together with different goals and desires and contributions to the common project. There is the hardware producer, the company or organisation that produces the OS, the software developers, the vendors of both software (“app stores”) and hardware, phone companies, organisations that dictate software and hardware standards and other regulatory bodies.

All of these parties contribute different things to the collaborative project of producing smartphone offerings. All of these parties have different goals. Many of these parties overlap. For example, in the case of iPhone app development, the hardware producer, the OS developer, and the hardware and software vendor are all the same company—Apple.

Similarly to the way that certain restrictions, practices or rules on human research help these stakeholders to work together (or not), there are also restrictions, practices and rules that make smartphone development ecosystems better or worse.

These are things like pricing structures for apps, “openness” of the platform, hardware limitations and consistency, etc. Often it is actually the restrictions placed on a platform that make it thrive.

Consider this article from the BBC about the Android platform.

Even my choice to develop for the iPhone was influenced greatly by the fact that there’s only one iPhone to write for, combined with the fact that distributing the app is not a huge headache—Apple takes care of the details. That’s good and bad, but the restrictions that Apple has placed on the iOS ecosystem seem to have been conducive to producing thousands of apps.

Getting back to human research, similarly, placing restrictions on human research might actually make human research thrive better. By getting rid of “bad” phase IV studies, universities can avoid situations like the one outlined here.

This whole situation is having huge cascading effects on phase IV research and human research generally at that institution, and could have been prevented. By taking a little bit more of a “walled garden” approach to human research, we can actually make human research work better.

BibTeX

@online{bgcarlisle2011-1748,
    title = {Human research ethics and smartphone application development},
    journaltitle = {The Grey Literature},
    author = {Benjamin Gregory Carlisle},
    address = {Montreal, Canada},
    date = 2011-05-16,
    url = {https://www.bgcarlisle.com/blog/2011/05/16/human-research-ethics-and-smartphone-application-development/}
}

MLA

Carlisle, Benjamin Gregory. "Human research ethics and smartphone application development" Web blog post. The Grey Literature. 16 May 2011. Web. 10 Dec 2018. <https://www.bgcarlisle.com/blog/2011/05/16/human-research-ethics-and-smartphone-application-development/>

APA

Carlisle, Benjamin Gregory. (2011, May 16). Human research ethics and smartphone application development [Web log post]. Retrieved from https://www.bgcarlisle.com/blog/2011/05/16/human-research-ethics-and-smartphone-application-development/

Like Times Square, but in Toronto

by

I went to Toronto this weekend, to work on a sequel to my Montréal Métro Exits app. It was fun to see the city and meet up with old friends.

Here’s a part of Toronto that looks like a mini Times Square. I’ve been there before, but never at night-time.

BibTeX

@online{bgcarlisle2011-1646,
    title = {Like Times Square, but in Toronto},
    journaltitle = {The Grey Literature},
    author = {Benjamin Gregory Carlisle},
    address = {Montreal, Canada},
    date = 2011-04-18,
    url = {https://www.bgcarlisle.com/blog/2011/04/18/like-times-square-but-in-toronto/}
}

MLA

Carlisle, Benjamin Gregory. "Like Times Square, but in Toronto" Web blog post. The Grey Literature. 18 Apr 2011. Web. 10 Dec 2018. <https://www.bgcarlisle.com/blog/2011/04/18/like-times-square-but-in-toronto/>

APA

Carlisle, Benjamin Gregory. (2011, Apr 18). Like Times Square, but in Toronto [Web log post]. Retrieved from https://www.bgcarlisle.com/blog/2011/04/18/like-times-square-but-in-toronto/

Montréal Métro iPhone app

by

Logo for Montréal Métro Exits

Logo for Montréal Métro Exits

On and off for the last little bit, I’ve been working on a little bit of a side-project: Something for when I don’t want to think about research ethics anymore. I was inspired to do this by something I heard on CBC a while back. A guy in London, UK made an iPhone app that would tell you which car to exit so that you would be closest to the exit on the subway.

I thought that this was a great idea. I would certainly use an application like that! Turns out someone already did it for Montréal, but they did a crappy job of it. The data set is incomplete, and the interface leaves much to be desired. Also, this other app tells you nothing about which car to board in order to transfer. In fact, the other app told you only which métro car to exit in order to be near the exit, not which métro car to enter, which seemed to undermine the point of the app. You need to know which car to board before you get on the train. (You can’t just infer one from the other, though, since in some cases the train approaches from the right side of the platform and in some cases it approaches from the left.)

I decided to write an app that would be really simple from the user’s perspective—just choose two stations, and the app tells you which car to get into at your departure station, and then which car to get into at your transfer station(s) (if applicable). I thought it would be a good exercise, just as practice for some other ideas for iPhone apps that I’ve had.

So, a couple weeks ago, I donned my lab coat, grabbed a clip board and went to every métro station in Montréal and wrote down where all the exits were. I also collected information regarding transfers. Writing the app wasn’t so hard, although submitting it to the iTunes store was a bit of a headache. That said, it was approved on my first try, and it took less than a week. (Thanks, Apple!)

It was getting Apple to process my tax forms that was the longest part of the development process.

The app was approved on Friday the 18th, and Apple processed my Canadian tax info last Tuesday. I had to fill out some US tax forms (just indicating that I wasn’t a US citizen) and then today they finally started selling my app on the iTunes store.

Tell your friends! Seriously. Every month I get roughly 300 visits to my blog from people in the Montréal area. If I could get a few of you guys to post this to your Facebook, I’d be raking it in. :)

Now that I’ve sort of figured out how to write and submit an app for the iPhone, I’ve got my sights set on bigger cities where this sort of app hasn’t been written before. (Yes, there are still some. Not many!) Also, I have a few ideas for other, better iPhone apps that I think could be a lot of fun. I’m not about to start posting my ideas on the internet though: That’s a great way to have someone else make my app before I do. :P

BibTeX

@online{bgcarlisle2011-1422,
    title = {Montréal Métro iPhone app},
    journaltitle = {The Grey Literature},
    author = {Benjamin Gregory Carlisle},
    address = {Montreal, Canada},
    date = 2011-03-31,
    url = {https://www.bgcarlisle.com/blog/2011/03/31/montreal-metro-iphone-app/}
}

MLA

Carlisle, Benjamin Gregory. "Montréal Métro iPhone app" Web blog post. The Grey Literature. 31 Mar 2011. Web. 10 Dec 2018. <https://www.bgcarlisle.com/blog/2011/03/31/montreal-metro-iphone-app/>

APA

Carlisle, Benjamin Gregory. (2011, Mar 31). Montréal Métro iPhone app [Web log post]. Retrieved from https://www.bgcarlisle.com/blog/2011/03/31/montreal-metro-iphone-app/

Search

Tag bag

Recent comments

Old posts

All content © Benjamin Carlisle