explore my world
explore my world encourages toddlers and preschool children to engage their curiosity about their surrounding world. This easy-to-use app has both an intelligent camera and an intelligent microphone, which can be activated to view the written words for your child’s surroundings and for the speech that they say or they hear. Each word is presented with its spoken equivalent if the speaker is on. If the speaker is muted, only the written word is shown. The words are zoomed out and presented in a large and eye catching format to attract your child’s attention to the words and, therefore, create a more vivid experience, which will reinforce their learning to read the words. This is similar to how the child learns spoken language.
The app should be only used in landscape mode with the home button on the right. It opens on a home screen, and requires an internet connection. It should be noted that all interactions with the app are completely anonymous, and the internet access is solely for the purpose of automated image and speech recognition. You can establish an internet connection in the Settings app on your device.
The home screen shows four buttons, which are guide, help, camera (see), and microphone (hear). Touching the guide button gives the instructions for using the app. The help button should be touched if you are having difficulty with the app. It will advise you to please consult the guide for possible help, or if you cannot find any solution, please email Dom Massaro at Massaro@ucsc.edu
The other two buttons on the home screen give pictures of a camera and a microphone. These two icons occur with the words see and hear. The descriptions of these two functions are given in the camera see and the microphone hear sections of this guide.
When the picture of the camera is touched, the device enters the camera mode, which is similar to the Apple camera with a white circle on the middle right for taking a picture. On the top right is a camera mode that switches between the two cameras. Touching the cancel button on the bottom right returns the child to the home screen. Control of the flash is given on the left side, and the default is the auto flash mode. The child explores their surroundings by looking through the screen of the tablet and searching for a scene to photograph. The child takes a picture by touching the white circle button on the right of the screen. Taking the picture puts the picture on the screen. On the iPad, the picture stays in landscape mode. On the iPhone, the picture now appears in portrait mode, due to a limitation of the Apple interface. With an iPhone, the child can either turn the phone to be in portrait mode or simple look at it rotated by staying in landscape mode. On both devices, the child has two options. Touching the Retake button puts the child back in camera mode to take another picture. If the child wishes to explore the labels for the picture, touching the Use Photo button accesses the internet to find word descriptors for the picture. The child should be in landscape mode to interact with the written descriptors for the picture.
These written words are presented one at a time. Each printed word for the picture is presented on the screen in large lowercase letters, and the word is zoomed out (made larger) to capture the child’s attention. The spoken word accompanies the presentation of each written word, unless the speaker volume is turned to mute. The written word and picture stay on the screen, along with large say and new buttons that allow the child to hear the word pronounced or to generate a new word describing the picture. If the child touches the large left button that says say, the word is presented again. When the child touches the large button that says new, another word is presented. The child can continue with this picture, generating new words. When the best five descriptors have been presented, the new button will recycle through the words once again. At any time, touching the large return button on the top left of the screen returns the child to the home screen. Touching the Apple’s home button leaves the program and returns to Apple’s home screen.
When the picture of the microphone is touched, the device shows a microphone and is in microphone mode. Touching one of the understand buttons changes the screen to red, which gives the instruction to start talking. The child, playmate, or caregiver says a phrase or sentence that will be automatically recognized. The screen also shows two done buttons, which should be touched when the person is done talking. The recording will stop after 15 seconds. The screen turns blue and presents the first word in the same manner as in the camera mode. The child touches the new button to see the next word and each additional word. By touching the say button, the child can have the word repeated. When all of the words in the utterance have been presented, the new button will recycle through the words once again. At any time, touching the large return button on the top left of the screen returns the child to the home screen.
It should be noted that the device does not permit any indecent or offensive words to be written on the screen or to be generated by the device.
The app is programmed on the Amazon Fire, which can be purchased for about $40 and can include a robust child protective case. The automated scene recognition is carried out by Clarifai, and the automated speech recognition by Houndify.