Visually impaired users and software development environments

What does it mean to be visually impaired?

From, The definition of vision impairment by the Centers for Disease Control and Prevention (CDC) says a visually impaired person’s eyesight cannot be corrected to a “normal level”. It may be said that visual impairment is the functional limitation of the eye or eyes or the vision system. Blindness is defined as a visual acuity worse than 20/400 with the best possible correction, or a visual field of 10 degrees or less.

Accessibility and the web

As Lazar, Dudley-Sponaugle & Greenidge write, “an accessible web site is very similar to an accessible building. An accessible building offers curb cuts, ramps, and elevators to allow a person with disabilities to enter and navigate through the building with ease. An accessible website offers similar functionality”.

Unfortunately, as the web grows faster and faster, most of these techniques are left behind when working on projects that need to attract millions of users in the least amount of time possible. The only associations that are required to publish fully accessible content on the web are Governments and big companies who need to face entire communities, such as Google, Yahoo and Microsoft as well as some of the major News Corporations.

One of the issues that might be related with a widespread development of accessibility on the web is related to the fact that, with the introduction of Web 2.0 that made everything way more visual than before, accessibility practices require a lot of additional coding in order to add descriptive content for each element of a web page. Detailed information about Web Accessibility Standards and ways of implementation are out of the scope of this research but can be found at W3C WAI for Visually Impaired users

This practice tend to get more complicated when web pages are loaded dynamically, with AJAX requests or other kind of implementation. More details can be found on the official W3C ARIA resource pages

Visually Impaired users of everyday software

A variety of solutions have been developed over the years to allow visually impaired users to interact with computers and technology in general. Most of them are software that read what happens on the screen to the users allowing them to then respond by inputting commands or text via their keyboard.

The major softwares used for this purpose are the Apple VoiceOver application, which is integrated in the operating system and JAWS and NVDA for Windows. The latter is the only one which is free and Open Source, developed by members of the visually impaired community for the visually impaired community. JAWS is one of the first screen reader softwares and also one of the most expensive solutions on the market.

Other I/O systems for Visually Impaired users are Braille Tactile Keyboards. However, according to this article only a small amount of the worldwide population can speak Braille fluently, which makes these solutions very hard to introduce in visually impaired people’s everyday lives.

Visually Impaired developers

As IDEs are becoming more and more sophisticated, it is becoming harder and harder for computer readers to identify parts of the IDEs and describe them to visually impaired developers. According to a series of interviews conducted bySean Mealin and Emerson Murphy-Hill at the North Carolina State University, visually impaired developers have developed skills that go beyond the use of a simple IDE to work on pieces of code.

In fact, many use external editors to keep track of all the variables they use in their code or to keep track of the flow and indentation of their programs. Also, most of them use “printf” debugging as major tool for understanding issues with their code as most of the existing debugging tools are not optimised for screen readers. Solutions like WAD (Wicked Audio Debugger) an auditory debugging system, have been developed by different universities. As reported by the team of researchers that developed WAD, “we found that participants comprehended the code with approximately 86% accuracy for our tasks”. Which shows how successful these solution can be and how they could facilitate developers’ life. However, they are not integrated in any of the most common IDEs and not yet available to the general public.

The following table shows a comparison of Java IDEs and how accessible they are. The information has been collected from Stackoverflow and other online resources as well as the official softwares documentation.


Design Exploration

The past couple of months have been a little crazy, and I had very little time to update this blog. But some things happened on the project I am working on for my thesis, and I want to post a review of the first stage of design exploration I have gone thru before and after the initial presentation of the project. As I mentioned in the previous posts, the project I am working on is a personal budgeting application which is trying to geo-locate expenses. The project has also a name now, it’s awear


In order to discuss the design process, I need to introduce you to a couple of previous works which brought me to following specific design and implementation patterns for the project. An excellent research, conducted by Kaye, Gulotta et Al (2014) showed how a group of selected people, interviewed in the United States, track their expenses and incomes in the most various ways – from paper, to spreadsheets, to specific application to a mix of a little bit of everything. Another helpful research paper is the one published by Li, Forlizzi et Al (2010) which explains the major issues related with how marketed technologies deal with life logging and monitoring our activities.

First wireframes

Starting from insights taken from those papers and personal experience with using existing application, I have mocked up an initial version of the interface. Since the main interaction with these kind of applications is numerical data input and visualisation, those are the two points I have focussed on for this early stage prototypes. Also, as the app is meant to come with a companion wearable extension, which will allow an even easier and immediate interaction with the application.

Initial mockups

After the first sketches, I have decided to just move on to something more “attractive” and to start designing some more realistic mock ups that I will later use to gather some feedback from the people involved in the development of awear. As I am designing an application that will be mocked on Android, I have decided to follow the recently released Google Material Design guidelines which are extremely helpful to develop applications that blend with the operating system and immerse the user in a great experience while using a kind of application that is not traditionally engaging or exciting to use.

Mockups V2

Since I didn’t feel those mockups were exciting enough, I have been inspired and decided to work on a new version of the concept. Everything is still based on circles, as coins are circulars and watches are circular too! What a coincidence… Anyway, I think the new version is the one I will go with for the implementation of the prototype. The idea of designing a new set of mockups came to my mind after I started using an Android Wear device. I have soon realised that my design wasn’t consistent and was not following any of the pattern I was getting familiar with on the small screen of my Moto360. The new design is much simpler to use, and way more intuitive. The following are the mockups for the “new expense” process.


Stay tuned!

Hopefully there will be more and more coming in the following weeks and months as I will start developing and I will keep keeping you up to date with my development process and research findings (when I will start having some!)

Phase I: Collecting Data

It’s been a month now since I have decided to track my physical activities and sleep patterns, using a Jawbone UP24 band. I wanted to see which sort of information a device like that is able to capture and why these fitness bands are extremely popular at the moment… Well I still have to figure out how to answer this question but I can say for sure that the platform (mobile app + arm band) didn’t really motivate me enough to keep on using it.

This experiment tho it’s been a great finding for my research, I will definitely need to figure out how to motivate people to keep on using whatever I’ll come up with or it’ll just end up being another app sitting in someone’s phone (or, hopefully, smartwatch) that won’t do much more than keep on collecting data and sending it to a server, somewhere in the cloud. Here are some of the major issues I had with UP:

It tracks your sleep, but what can you actually do with that data?

2014-09-14 23.20.41

It tracks your steps… Ehm… Not really, it tracks every (or almost) movement of your arm. Now I know I move my left arm about 15000 times a day, I accumulate heaps of steps while cooking, coding, showering…! However, stripping off about 7000 steps a day, it becomes sort of accurate. The app tells you when your peaks of motion are and it’s fairly easy to find a pattern over a week – walk to work, walk to lunch, walk back to work, walk back home.

2014-09-14 23.21.06

This looked like it was going to be an exciting and interesting feature, but apart from telling you how many calories you burn during those walks (which are probably not exactly accurate, because it doesn’t have any GPS module and doesn’t know whether I’m climbing a hill under the 30° of Queensland sun or walking in a mall…) it also supports user input for various sport activities. Interesting feature, which has been terribly designed in the app and it’s pretty hard to find (about 4 clicks through the interface) – I gave up after a couple of times.

Oh, it also tracks your mood. Based on self reporting, you should remember to input every morning if you’re feeling Good, Energized, Depressed and so on… But that’s certainly not something that everyone would remember to do in the morning… So I ended up not doing it anymore after the first week…

2014-09-14 23.22.47

And, last but not least, it tracks intake calories. At every meal, you should tell the app what you’ve just had and it’ll automatically try to calculate how many calories were in your food and how distributed (in terms of carbs, proteins, etc…) your meal was. This is probably one of the most useful features of the app, especially if you’re on a controlled diet.

It’s an interesting experiment, especially rolling back in time and seeing what happened on specific days. It makes you feel like you should follow some patterns so that you can then compare them over time… But on the other side it doesn’t exactly give you any info on how you could do better – there are some tiny notes that are related to your lifestyle in the app, but I have only received four, and they never changed over time. It also allows your analyse your trends, which can be useful if the band was a little bit more reliable as it could be interesting to see how physical activity changes between days where you get long sleeps and days when you don’t, or something like that…

2014-09-14 23.23.15

There’s, however, an interesting aspect related to the data I have collected: it is stored somewhere and can be access using the Jawbone API. I believe that way is possible to kind of filter out “fake” steps and see what the actual walking / exercising patterns are and probably generate some reports that’ll help the user to engage in a more active lifestyle.

I would be curious to find out more about your experiences with Jawbone or Fitbit or other similar devices, feel free to leave a comment if you’d like to :)

Information Design by Robert Jacobson

This book gives a clear overview of how information can be helpful to designers and which techniques designers should apply to make information meaningful and useful to the crowds. In particular, a chapter of this book captured my interested, Presenting Information by Jef Raskin. He discusses the terminology used in the field of Information Design by proposing a different approach towards the analysis of the subject.

“Information is an abstraction from any meaning a message might have and from any particular form a message might take.” (Jef Raskin, 2000)

In his analysis, Raskin tries to explain how the difference between information and meaning needs to be clearly distinct as practicing information design is about giving meaning to a set of data, or information.

Find out more about Jef Raskin and his books.


Thoughts on IxD by Jon Kolko

As I was going through the selection of articles and books I was using in support of my research process, I came across Jon Kolko’s Thoughts on Interaction Design book, a useful resource to get full on into IxD. I have enjoyed reading bits and pieces out of his book and I found the approach he adopts creative and diverse and I have found his book way more engaging than any other Interaction Design publication I have read so far.

It’s worth giving it a try – for more details, have a look at his website:

I’m back!

Alright, I am back. With a bunch of new exciting ideas and lots of stuff going on.

I am going to use this space to talk about my thesis project. I have recently started to research in the field of information and data visualisation which is a field I have always found extremely useful and not very well explored, especially when it comes to data mashups and giving meaning to data analysis. We are filling up our world with digital gadgets that do (almost) everything we want and need by generating huge amount of data that we’re still uncertain of how to use and analyse.

This video, from the CIID (Copenhagen Institute of Interaction Design), is the outcome of a research project called “Quantified Self” which was looking at how we can visualize, in 3D space, information related to our sleep quality.

Another examples of alternative use of data, in this case data collected by using a mixture of FitBit trends and measurements and user feelings throughout a day, is the IAMI project, a curious and attractive way of generating a portrait.

During the development of my project I will try to look at how we could record and analyse specific patterns related to our lifestyle (sport activities, mood, workload, etc…) and merge them to our “personal financial world” (which will include mainly expenses and income), to create a budgeting platform that adapts to ourselves and gives us a real picture of how we manage our finance.

Keep tune for more updates to come, because this time I am not going to disappear!

Hello, Again!

Hi everyone!

It’s been ages (seriously ages!) since the last time I sat down and wrote something on my blog. And I was getting extremely bored of the old and plain (super plain!) design I chose while I was traveling that the inspiration just went away…

But now I’m back, and I promise this time I won’t leave anymore – for some time! I have been thru a LOT of exciting and interesting stuff during the past 6 months, tho whole traveling from China to Australia by bus, boat & train; the landing in Australia with NO idea of what to do and how to survive in this new World; the enthusiasm of starting a new and fascinating University here in Brisbane… Everything is still very wow~ish!

As I said, I will keep this blog up and running for the next ( whatever period of time ) and I will use it as a journal of my creative-experience (!?) and my world-exploration around here and who knows where else!

A couple of words about the new name: Cook and Hack. I have no idea of where it actually came from… I have always enjoyed cooking, and hacking (in which ever sense you want to interpret hacking – from computer code and hardware, to braking vacuum-cleaners!) so… Why not putting these two things together and have them working together: why cooking should be different from a computer built up from scratch? And why should a million lines of code software be different from a recipe to bake something? Let’s see what is gonna come out of here!

I will say goodbye for now by attaching an amazing video I’ve found a couple of days ago. Have fun!!