I have been working last month on testing our mobile application and trying to deliver the same. We have native mobile application, which makes use of SMS and GPRS to interact with central server on the cloud.
1. Lack of physical devices: We are building for j2me and android applications. How do i make sure that the application works on different permutations of android and j2me phones in the market? In India, we take the help of mobile stores that sell second hand mobiles to test on variety of physical devices. We also engage our friends, ex colleagues and family to perform testing. But we always question what is quality that is good for our friends to spend their personal time free for our application testing.
2. State of connection for QA testing: When the application uses SMS, we have to really send SMS to test the end to end functionality of the application. The emulators cannot simulate SMS sending and receiving in the local developer or QA environment. it was a great relied with GPRS connection on native j2me application and native android application, as they provide their emulators a mechanism to connect to the web server running on local-host.
3. Scale testing and pilots: For a test engineer to use emulator and perform testing multiple times is not easy job. In pilots too, there is a good amount of effort spend to educate students to find where the application is available on the mobile and launch for the same. I see that this challenge reduces with android phones and high end j2me phones.
All of us remember our passwords and mobile numbers. But when we use mobile number as identifier in the application, it is difficult to remember multiple identifiers. To add with the mobile number when used as identifier is 10 digits long and is not easy to remember multiple of them during testing.
4. Time: We see that application developers choose to hit ‘fast forward button’ on their testing process, as it occurs at the end of the cycle. Small glitches, crashes and malfunctions are overlooked.In addition once developer are used to error, it never occurs that there is a better way to fix the same and they start to live with the same.
5. Reliability: we try to store all message received from the mobile on the server prior to processing the message. This helps us to make sure that message from the mobile has reached the server and also to be aware of the capability of the mobile used to add this part of our test configuration. There is a long way to go here.
6. Different Mobile Platforms/OS: There are different Platforms/OS currently present in the market. Android, iOS,J2ME phones, Windows 7, Blackberry (RIM) and so on. Diversity in mobile platforms, different OS versions and platform limitations make it a challenge for testing mobile applications. The challenge is increased as there is more inconsistency in terms of functionality across multiple devices of same platform and every platform may have some limitations.