The folks behind HackYourPhD (see last post) are wrapping up their US tour with an online event highlighting their work over the summer, interviewing key players in open science.
I’ll be joining the Hangout, along with William Gunn (Mendeley), Brian Glanz (Open Science Federation), Jeff Spies (Center for Open Science / Open Science Framework), Charles Fracchia and Adam Marblestone (Biobright), and Ann Lam and Elan L. Ohayon (Green Neuroscience Lab).
Mozilla Science Lab and HackYourPhD
A few months ago, I heard about a crowdfunded project called “HackYourPHD” – a community and resource building effort led by a researcher in France to collect information and run interviews with a diverse cast of characters in the open science world. The founder, Celya Gruson-Daniel (a neuroscientist by training), had become frustrated by many of the analog practices and mindsets in science, and decided to set out to crowdfund a trip through North America to do research on open research.
I had the pleasure of chatting with Celya last week at Columbia University, as she passed through New York on her nationwide tour. You can listen to the audio of our interview linked above, as well as listen to her interviews with other key folks in this space here: https://soundcloud.com/hackyourphd/
The second of P2PU’s Open Science course videos is now online, this one focusing on “Open Data”. I was delighted to be able to join this module, alongside course-master Billy Meinke, Heather Piwowar from Impact Story, Ross Mounce from OKFN and Wouter van de Bos from Max Planck. You can check out more of the video modules here: https://p2pu.org/en/courses/5/content/1370/.
Have a watch, ping us at @MozillaScience with any questions (or leave them in the comments), and if you’re feeling *really* creative, try remixing / splicing the video using some of the Webmaker tools:
https://webmaker.org/en-US/tools/#popcorn-maker
And let us know what you come up with!
Our new code review pilot is featured at the start of BBC Radio 5’s show “Outriders”. Have a listen. We’re right at the top of the podcast.
The life sciences would seem, on the surface, ideal for open source. It’s a world built on disclosure – whether publication or patent – it doesn’t count until you tell the world. It’s a world where the knowledge itself snaps together in a fashion that looks eerily like a wiki, where one person only makes a small set of edits in an experiment that establishes a new fact. And it’s a world where the penalty for redundancy is high – no one in their right mind wants to spend scarce research dollars on a problem that has been solved already, a lead that is a dead end, a target guaranteed to lead to side effects.
– John Wilbanks in his recent Xconomy piece, “Understanding Open Science”.
Over the years, science has evolved many quality control mechanisms, the best known of which is probably peer review. This first appeared nearly 300 years ago, and was initially put into place to allow trusted peers to scrutinize new findings, ideas, and societal implications as part of the publication process.
Here at the Mozilla Science Lab, we’re interested in finding ways of pushing the limits to what we think of as “science on the web” and instilling better practices for open, reproducible research. We aim to do this through community building, skills training (like Software Carpentry), and building prototypes and running pilots with other groups and organizations.
Our latest experiment is exploring a model for code review in science. As research becomes increasingly dependent on computation, we want to see whether the code review practices that are now commonplace in open source software development will also work in research.
Why code review?
The code used to produce the results in a paper is not usually reviewed when a paper is published, beyond a basic “sense-check” (or, as a colleague put it to me, “plausibility versus defensibility, not reproducibility”). As code follows the trajectory of data in being integrated and recognized as a first-class research object, we want to work with the scientific community to figure out how scientists can and should check each other’s computational methods without increasing the time and cost of reviews unsustainably.
How this will work
The staff at PLOS Computational Biology helped us identify a set of already-published papers that include snippets of code in MATLAB, Python, R, and C++ that may be suitable for code review. We’ve put the code in front of a group of Mozilla engineers who regularly do code reviews for Firefox, mobile and for other programs.
These engineers aren’t scientists (though some may be interested in the underlying research). Instead, they are skilled professionals who don’t have the domain expertise of the scientists, who in turn are not full-time software developers.
Over the course of the next month, each Mozilla engineer will review code from one or two papers. Their reviews, and their insights into the review process, will be captured in ReviewBoard, so that we can have a core collation point for the findings. Reviewers will have the ability to flip through the rest of the paper should they so choose, but this is really about the code snippet itself.
Once they’ve finished reviewing, we’ll analyze the findings, look for any patterns or interesting outliers, as well as examine how the actual commentary on the code itself was done, with the aim of using that as a model for code review in the future.
What we’re testing
This experiment is a means to explore the following:
– What does code review from a software engineer outside of academia look like? How do they approach the task?
– To what extent is domain knowledge needed to do a successful code review? Is the code parseable by someone outside of that discipline?
– What lessons can be learned about code review, possibly to influence and enhance traditional peer review?
– Does this process surface issues around best practice in writing software and code? If so, what are those issues?
– Following the review, how useful is the markup to the author? Does this feedback help them in their work? How can we change those norms?
We’ll be writing more about this experiment as the reviews continue, and hope to have our analysis completed next month. It’s just a first step, but we believe that this could help change the way researchers, administrators, publishers and funders think of code in scholarly research, and possibly unveil some deep-seated issues surrounding how code is recognized, maintained and shared in the scientific community.
—
Have a question or would like to join in on the conversation? Drop us a line at [email protected] or add your comments here. We’d love to hear your thoughts. Also, have an idea you’d like explored. Let us know.
And special thanks to our Mozilla volunteers, Marian Petre from the Open University, Mike Hoye from Mozilla and Greg Wilson from Software Carpentry.
The first of P2PU’s Open Science course videos is now online. (I’ll be joining to speak on Open Data in a few weeks.)
Have a watch, ping us at @MozillaScience with any questions (or leave them in the comments), and if you’re feeling *really* creative, try remixing / splicing the video using some of the Webmaker tools:
https://webmaker.org/en-US/tools/#popcorn-maker
And let us know what you come up with!
The Mozilla Festival (or “MozFest” as we affectionately call it) is coming to London October 25-27, and planning is in full swing. This year we’ll have an entire track on “Science and the Web” and we want you to help us shape the program.
MozFest is where many of Mozilla’s best and most innovative ideas come from, bringing together over 1,000 inventors, hackers, creatives (and now those from the research community) to share ideas and explore how we can forge the future of the web together.
For the “Science and the Web” track, we’re looking for sessions that show how the Web can (/and is) transforming science, and help introduce the MozFest crowd to some of the folks and organizations leading the charge. This isn’t your average conference, where you present to a group. Preference is given to hands-on and collaborative sessions (in terms of groups involved as well as the session content itself). Or, as we say “Less yack, more hack.” 🙂
A few areas to explore as a starter, in case you’re looking for ideas:
This is just a start, and feel free to stray from this list. We’re currently pulling together the program, and the session call is open through the end of August for you to submit your ideas. Check out last year’s program, and get those proposals in! To submit, fill out this short form (and be sure to select “Science and the Web” track.)
We look forward to your submissions, and don’t forget to get your tickets!
From Selena’s presentation on PyLadies and what the open source community can do for K-12 teachers.
Daresay this is also applicable for the researchers and educators we work with at the Science Lab …
I recently came across a web-based visualization project called “Luminosity“, a recent winner of the Mozilla Ignite app challenge led by Amit Kapadia, a developer on the citizen science project Zooniverse. Ignite is an open innovation challenge hosted by Mozilla and the National Science Foundation (and a sister project to the Science Lab), funding app development showing the potential of gigabit fiber networks.
Luminosity sets out to better enable researchers to create scientific visualizations based in the browser. The app is focused on astronomical data, exploring the power of using the web as a platform for the sciences.
At the Science Lab, making the web work for research is in part about creating open, innovative tools to help advance discovery. Sometimes these tools are open source by design, othertimes not, but still contribute nevertheless to open science.
Amit’s approach with Luminosity takes that work to another level, exploring using the browser as the platform – a space Mozilla is quite involved in 😉 – to create scientific visualizations. I asked Amit if he would share his thinking about the possibilities for not only better visualization tools in the sciences, but also about his design decision in making his app browser-based.
Kay Thaney: Thanks for joining me today. Let’s start with Luminosity. Tell us a bit more about the problem the app sets out to solve, and how you address it.
Amit Kapadia: Luminosity is an experiment to bring science grade tools to the web. The browser is evolving into a full-fledged platform where all types of applications can exist. Many software applications in the sciences rely on older technologies, often sitting on a 10 year old code base. Luminosity is an attempt at a fresh start. In its current implementation, the app allows astronomical data to be imported and visualized in the browser, emphasizing a modern user experience, performance, and a collaborative workflow.
Kay Thaney: One thing that struck me from the description was the web-native design. In today’s world, most of us interact with the web through browser-based applications. Is Luminosity’s design unique when it comes to the sciences, or are others taking the same approach towards data visualization tools for research?
Amit Kapadia: The functionality of Luminosity is typically expected from a native desktop application. Putting aside the science for the moment, there are many application such as Photoshop or Final Cut Pro, that we know as being strictly desktop applications. Over the last years, browser technology has advanced tremendously such that functionality of these heavy-weight applications can be executed in a browser. Proof of concepts are beginning to emerge, but far fewer in the sciences. One example from the sciences is Slice:Drop, an application for visualizing 3D medical data.
Kay Thaney: Did you encounter any challenges in taking a browser-based approach? Why aren’t more startups and tool developers in this space creating tools in this manner?
Amit Kapadia: There are difficulties taking a browser-based approached. Many disciplines, such as astronomy, have existing toolsets that have matured over the years. These are often well established libraries written in low level languages. Migrating to the browser requires porting these libraries to JavaScript, so that, for example, scientific formats can be read in the browser. This is a difficult task, but offers the reward of using the browser as a scientific platform.
Only now are developers beginning to utilize the full power of modern JavaScript engines. The gap between native and JavaScript performance is narrowing thanks to the large investment of Mozilla and Google. It would serve the science community well to take advantage of this technology. Many developers are still not aware of the level of performance. Hopefully that changes, and we’ll see more desktop applications migrating to the browser.
Kay Thaney: At the Science Lab, we’re strong supporters of providing not only better tools but ones that can help make concepts such as data visualisation more accessible and understandable to researchers. Looking beyond astronomy, do you think Luminosity could be applied to another disciplines data problems (say, ecology or the life sciences)?
Amit Kapadia: Many of the initial ideas behind Luminosity have been specific to astronomy. Currently the web app only supports a file format known as FITS (Flexible Image Transport System), a format archiving nearly 40 years of astronomical data. Despite being able to read an arcane format, the real problem that Luminosity addresses is visualizing complex imagery. Astronomical images have a very high dynamic range, requiring tailored tools to best visualize the data. Medical and satellite images share the same problem, storing their data in formats such as DICOM and GeoTIFF. In the future, Luminosity could be extended to offer more domain specific tools, while levering it’s current capability of displaying high dynamic range images. The plotting features, utilizing D3, are already generic. Other disciplines could take advantage once other file formats are supported.
The most experimental feature of Luminosity is the collaborative workflow. Collaborative image viewing has been implemented for a subset of data, allowing a group of researchers across multiple institutions to view and manipulate an image together. Collaboration is the most straightforward idea to apply across different disciplines, including education. Cizzle, another recent submission of Mozilla Ignite, is developing a web app for students to better understand our Solar System. Between these tools, one could imagine researchers digitally visiting a classroom for a data exploration activity.
Kay Thaney: What is next for Luminosity?
Amit Kapadia: Luminosity is still experimental. In the short term I’d like to continue building collaborative features, so that researchers spread across a large distance can visualize data together. Another item on the to do list is visualizing three-dimensional volumes, and luckily WebGL provides a solution. The application will continue to utilize only open standards, and I’ll continue to experiment with ways to approach performance of native code.
Kay Thaney: Thanks, Amit. We look forward to hearing more about Luminosity’s development in the future.
Want to check out the code behind the project? Visit Amit’s GitHub page and have a look. You can also see the tool in action in this video.
kaitlin thaney 