Automation of Mobile Testing – Innovation or a necessity?

By Vaibhav Sharma No comments »

It was 10 pm and after waiting patiently for a build to test the latest release delivery for a complex app – this was to be the 4th of 5th delivery drops for this app – one with 3 new functionality elements but a fairly complete and functional app. Client delivery weeks are generally crazy but I was getting tired of the waiting game. I have always appreciated the hard work my development counterparts put in to create, troubleshoot and refine code but after 3 years of being a test engineer and having gone through these wait periods multiple times, I have concluded that waiting is boring.

So instead of being frustrated and getting on the development teams’ nerves, I thought of doing something creative with what was available to me – test scenarios, scripts, use case documentation, functional spec and an app with 3 builds already functioning well with <10 bugs. I decided to automate a suite of build validation tests (BVT). I always was of the opinion that automating testing for a mobile app is a bit more complex than for a non-mobile application; but I decided to try and prove myself wrong.

Here is what I researched…

In pursuit of automation, I had to benchmark my current efforts and compare them with efforts needed for test automation. For manual testing, the efforts usually multiply in linear manner with every build and new features. However, for automation it was obvious that there will be a more upfront cost which will provide cumulative benefits with every subsequent release. Here’s how I put the automation efforts and payback.

click for larger image

While there will be initial high effort for scripting the automation tests, most of my test automation effort will curve down and will benefit for future regression. I was looking forward to the day when I would be able to conduct unmanned test execution (CUTE :) ).

click for larger image

I had the following expectations from my automation activities:
1. Handle device diversity involving multiple platforms and app runtime frameworks.
2. Ability to automate testing of web based hybrid apps in addition to native apps
3. Address Application rendering differences on different platforms
4. Support device simulators
5. Keep the test-code and app-code Separate
6. Be cost-effective

Here are the reasons for my selection

If my project budget had a dedicated budget for Test Automation, I would have gone for SeeTestAutomation from Experitest which provides extensive documentation and support. As my endeavor for automation was a bootstrap project, I decided to forego the license spend and experiment with something which was available for free or at a price-point which could be justified as ‘throwaway’

So onto the drawing board, I had the possible tools and the requirements: organized as below. I tried to play devil’s advocate here, to stress on the demerits and their impact

click for larger image

While building a case for optimum tool, one has to know that if there is an Agent to be embedded in the app then it needs to be removed before submitted to Apple App Store, which means to assure the app is Ready-to-submit (RTS) one has to perform a final manual testing. So considering this and other offerings, Appium and Calabash seemed to impress me, as they were the ones that fit within my constraints and yet fulfilled my objectives. At the onset, I chose Appium as it offered support for multiple languages rather than Ruby being the only choice in case of Calabash. This was a personal choice as I know Calabash to be an extremely capable tool.

With the project delivery drop number 5 three weeks away, I collaborated with another teammate and decided to spend a couple of hours each day to work on the BVT automation. We selected test cases and ran them by the project’s business analyst. It turned out that she too was in a similar waiting game and appreciated the extra effort we were putting in. A week later we were ‘testing’ our automation suite of 30 test cases and here is what we found:

click for larger image

I had the following expectations from my automation activities:

Our initiative had turned out really well. I showed this to my manager and was even more encouraged to see his excitement and support. He was able to get me some much needed help in a couple of areas which I struggled with. These were.

1. Support from a senior developer to help with scripting support for hybrid applications
2. Stakeholder input to identify and segregate test cases for automation
3. System Admin support for running test cases during off-hours
4. Error handling during test case execution, so as to continue the batch of tests even after an error is found.
As it turned out, 6 months from when I automated the first BVT suite for my project, this practice of automation has become the norm for every project in the company. Our team has grown and we are now adding mobile test automation as a service line to our portfolio.
Here is a generic architecture of how we have tackled mobile test automation.

click for larger image

I had the following expectations from my automation activities:

In my opinion, automating a regression test suite can have provide immense benefit when tests can be executed off-hours. This may not be possible with every project and the constraint of environment and test data availability, however I firmly believe in the real benefits which can be realized. I want to share some statistics with you on the productivity benefits this has brought us as testers, our company and our clients:

click for larger image

With these benefits, I would say that test automation is a necessity for every development project. Our experience with automation in mobile testing has been so effective that we now consider this as a key contributor to increased productivity, better quality and effort reduction.

As a company, the test team now works even more closely with the development team to be future-ready for new OS updates and development frameworks. My experiment has blossomed into a full blown practice area and we have incorporated mobile app testing into all our engagements.

The author is a Sr. Test Engineer at PopcornApps. His passion and initiative has led the company to develop a practice and we now have capability with a wide range of test tools available in the market place.
Are you losing out on productivity and quality?

To know more about our services, please fill out the information and you will get a call back from one of our associates.

Enterprise Mobility – the whole 9 yards

By Samvit Raina No comments »

Enterprise Mobility is a sexy term these days – Technology analysts, software companies, service providers and cloud providers are all using this term. It is rare to see the hype not living up to the reality but, in this space reality is far exceeding the hype.

While there are a lot of people talking about Enterprise Mobility, views on what this space stands for are in silos; Some think Enterprise Mobility to be overly relevant to BYOD, others feel it has a lot to do with security on mobile devices used within the enterprise, and yet some others feel it relates to employee access to enterprise applications through their smartphone or tablet. At PopcornApps, we believe Enterprise Mobility is all of this and then some!

Our definition of mobility in the enterprise

Deploying enterprise process(es) on a secure and reliable mobile channel accessible to employees, customers and/or partners, enabled through a app or browser without the need for the user to use anything else but their smartphone or a tablet device.

Based on this definition, we believe the enterprise mobility space encompasses Mobile App development, Security, Business process enablement, role based access, integration to enterprise and 3rd party applications, User Interface design, User Experience design, Analytics & Insights, integration to Device and App Management platforms, testing, and application deployment to public and private app stores.

Many aspects of what is considered traditional IT – enterprise software development, hardware/cloud infrastructure, APIs, integration buses, user security, SSO, load balancing, testing should be elements of a well thought through Enterprise Mobility initiative.

As might be evident, the mobility journey within an organization is not about one app, it is about a portfolio of applications which cater to various constituents and stakeholders. Following the key considerations listed above will help you realize the following:

Effective ROI Tracking

ROI for a mobility project should be looked at with the following criteria in mind:

  • Cost savings through use of the mobile app
  • Employee productivity enhancements through reduction in paper based processes, reduction in input errors and time savings on employee actions
  • Improved business metrics – sales conversions on campaigns, increase in revenue/profit, higher customer satisfaction

Analytics & Insights

We recommend building analytics into the apps – not only to track app usage and in-app user behavior but also capture business metrics. For e.g. time to complete a workflow/function, app function usage categorized by user/function. Analytics will provide you insights on improving key business metrics.

Not just one app

Mobility is here to stay and the change organizations have to embrace is ‘Think Mobile First’ and remember this is not about one app but likely an app portfolio.

Here is our holistic approach on deploying mobility solutions in your enterprise

 approach on deploying mobility solutions
Like what we have to say?

PopcornApps introduces smartMOBILE for Utilities

By Samvit Raina 1 comment »

At PopcornApps, our focus is on a few industry segments. For these industry segments, we build solutions to address a specific business objective or problem common to the industry. One of the segments in our focus area: Electric & Gas Utilities.

For Utility companies, we are proud to introduce our smartMOBILE for Utilities solution.  This solution provides 3 layers for a mobile application (Functional, Integration and Data), which work to deliver a comprehensive set of functionality to the customers’ smartphones.  For e.g. functions like ‘My Usage’, ‘Alerts & Notifications’, ‘Rate Plans & Goals’ are readily available to the customers. This solution allows for interactivity by enabling actionable alerts and notifications related to the customers’ usage patterns, outages or account related issues.

click for larger image

Advanced Metering Infrastructure and in particular, Smart meters, have become change agents for the customer relationships of a Utility company.  Smart meter deployments have reached a critical mass and continue to grow globally.  Utility companies fully expect to leverage smart meters to roll out energy efficiency programs –  Presenting usage related analytics to their customers has been a good start.  Most Utilities provide rich data to the customers over a Self-Care or Account Management web portal.  This is accomplished through a “Pull” mechanism i.e. the customer has to login and review information on a periodic basis.

PopcornApps’ smartMOBILE for Utilities solution increases the level of engagement a Utility company has with its customers by providing a “Push” approach to information delivery and completing the functional loop with response notifications back from the customers.  Such a response could be used to enroll the customer in a Dynamic Rate plan, Change the Alert threshold or simply provide more information.

Once deployed, this solution will drive adoption of the mobile app, thereby allowing Utility companies to gain a better handle on usage patterns, generate customer and regional usage profile, and ultimately offer valuable input into generation.

Mobile OS Platforms rule – Apps must allow deeper OS Integration

By Samvit Raina 4 comments »

In the recent past, we have seen mobile OS announcements on version upgrades/updates from Apple, Google and Microsoft.

Apple announced iOS 6 at their World Wide Devleopers Conference (Jun 11 – 15, 2012)

Microsoft announced Windows 8 Phone and Windows Phone 7.8 (to support current devices) at their Windows Phone Summit on June 20, 2012…

…and today, June 27, 2012, Google announced Jelly Bean (Android 4.1).  All these OS have new features – some which will be considered innovative and others incremental improvements.

I skimmed through most of these announcements and later went back to look at the details on a few features of each OS.  Each OS update has enhancements specific to their “own” offerings; Apple has new Maps, Microsoft has deeper Skype integration, Google has Advanced Search.  However, my observations lead me to the conclusion that all OS platforms want to become a one stop shop for consumers:  Not only when it comes to content consumption but also to useful, transactional functionality for every day use.  Let me elaborate by highlighting a few features…

Wallet Features:

Passbook in iOS / Wallet Hub in Windows Phone / Google Wallet

The similarities in Passbook and Wallet Hub are stunning and since both are yet to be released, we will have to wait and see which is more elegant and provides ease of use.  While all 3 can take advantage of technology like Near Field Communications (NFC), Google Wallet seems to be more aligned to it and is banking on new payment peripherals at retailers.

Microsoft and Apple are hoping to leverage updated barcode scanners, which can read smartphone screens.  A lot of retailers are already deploying these scanners.

What does this mean to businesses? 

As an example, Starbucks has already deployed smartphone apps to manage their loyalty card; well it is more than just a loyalty card – it is a mobile wallet that supports Starbucks as a retailer. Users can “load” their Starbucks card into their phone and the app takes care of account management, payments, loyalty points and promotions. The app has been hugely successful and since Starbucks was unwilling to wait for NFC to mature, it has created a new standard of sorts for mobile barcode integration. 

Starbucks may be considered a leader here, other retailers who are still contemplating deploying mobile apps (or upgrades to existing ones) may be able to save time and money by leveraging features like Passbook and Wallet Hub and integrating them into their mobile apps.

Please note that in addition to account management functions, these features can used for other transactional integration – a mobile boarding pass your airline, a movie ticket, a promotion coupon – you get the gist…

Social Media Feature:

iOS 6 Facebook integration, Windows Phone People Hub (Currently available)

iOS 6 will bring functionality that has been already present to a large extent on the Windows Phone platform.  Once your Facebook account credentials are made available to the OS, default OS applications like Calendar, Contacts and activity feeds will be seamlessly integrated.

Windows Phone takes a different route to social media integration. The “Emails and Accounts” menu under Settings of the Windows Phone allows you to specify account details for Facebook, LinkedIn, Twitter and Windows Live in addition to your Exchange, POP or Google Mail.

Once these accounts are set up, a swipe to the right in the People Hub (aka Contacts) will provide a news-feed which allows for viewing “What’s new” in all accounts or can be filtered by Facebook, LinkedIn, Twitter etc.   Additionally, upon selecting a particular person (contact), a user can view updates, pictures, posts, tweets and their own interaction (phonecalls, emails, sms messages) with that person.

 

 

With this deep integration provided by these OS features, the need for a native Facebook app is in some ways eliminated – 

Good news from a users’ perspective but bad for Facebook because it loses the opportunity to present ads.

 

 

Real Time Feature:

Google NOW / Windows Phone Live Tiles

Google NOW is going to be a feature in the new Android 4.1 OS dubbed Jellybean.   This feature gathers information from the phone and users’ previous behavior related to search, calendar, travel plans, likes and dislikes.

An example provided is a user’s commute pattern – any possible disruption to it would be highlighted in a “card”  – such cards can be resident and updated live on the home screen.

Windows Phone Live Tiles work in a similar manner but rely on the underlying app to support the ‘update’ aspect.  For example, a user can have an American Airlines app pinned to their home screen.  Based on the travel schedule, this ‘tile’ will have flight information like Terminal, Gate, or delay in departure or arrival of the flight.

With these enhancements, the OS platforms will continue to become more real time and integrated to provide everyday features for users.  I don’t think the app marketplaces will see any downturn in the near future but apps will provide a rich integration APIs to ‘blend in’ with the OS.   As far as ad related revenue, there may be a shift coming in how these apps promote products and services.

Enterprise Mobility by PopcornApps

By Samvit Raina No comments »

Over the last few years, that smartphone in your pocket has become an extremely useful device – to the “Don’t leave home without it” point.  The advancement in mobile technology, networks and the availability of applications has enabled a progressive on the go behavior.

But I submit to you that mobility and its uses have only begun to scratch the surface when it comes to enterprise applications, the ones that fulfill a set of useful business functions.  There are organizations emerging as leaders in early adoption with innovative set of features – for example: Chase Bank has an application which allows customers to take a picture of a check using their iPhone’s camera and have it deposited to their account.  There are several other applications, which blend in enterprise application functions with mobile device features quite nicely.  To me the whole idea of writing a check is obsolete but  I am sure Chase and other banks will do a lot more to change consumer behavior over the next few years.

 

Over the next several years, mobility will change drastically – the technology evolution in smart devices, the growth of tablets and increased affordability will allow user adoption of smart devices to exceed feature phones globally…. But the main advancement will be in usability and availability of applications, which enable end-to-end business processes on the go!

Mobility and everything to do with mobile devices has been a passion of mine for years.  I have always believed that success comes in loving what you do for a living.

Focusing on my passion for innovative technology and love for mobile devices, my venture PopcornApps provides Enterprise Mobility as a core service offering.

PopcornApps is on a mission to help clients enable a rich experience for B2B or B2C users of smart devices; providing functionally relevant business processes optimized over the mobile device channel.  Integration to existing applications for functions like Product & Service catalog, Billing & Payments, Order Management, Shopping Cart, Work Orders, Trouble Tickets etc. is the real key to delivering a good enterprise mobile application.

My team and I are working hard to bring meaningful solutions to a myriad of industry segments and geographies.  I look forward to the support of my personal and professional circle for guidance, advice and enabling business growth for PopcornApps.

This infographic, a first of many provides a view into the core service offering from PopcornApps.

Desperate for differentiation

By Samvit Raina No comments »

Competitiveness in the market place is an ages old phenomenon (well maybe not ages – Webster says the first known use of the word ‘Competitive’ was in 1829) – anyway in my book that is ages old!

Competition has been a factor for individuals (in sports, studies, debate, politics etc.), for countries (Super power, leading economy etc.), for service providers (Hotels, Airlines, Telecom operators, IT companies etc.) and for products.  It is a phenomenon that gets sparked when an innovator brings an idea to fruition and others try to ‘ape’ the idea or ‘better’ it.

As competitors compete, it becomes important for them to differentiate and highlight how their product, service or solution is better (or different) than their competition.

A company that has managed to differentiate its products and services is Apple.  Apple has been at the forefront of innovation to the extent that many a strategy session have asked the question: “What is your ‘iPhone’?”.  Well the iPhone, iTunes, iPad have truly been game changing products and services and have been aped by Apple’s competitors.

Apple has an integrated model of product development with a closely managed supply chain and a marketing machine which keeps the products ‘edgy’ and consumer oriented with a great deal of geeky stuff (like processor speed, chipset jargon) purposely kept out of the ‘spec’ sheet for the most part.  Apple also closely controls things like the Operating System, hardware and related third party software used by its products.

But I submit to you that in this fiercely competitive and fast moving world of Telecom, the differentiation between players is fast eroding and while there have been several products touted as ‘iPhone killers’, some of products on the market today may come close in form and function.

The iPhone has created competition in the handset/mobile device space and players like HTC, Motorola, LG, and Samsung have partnered with Google and Microsoft to provide choice to folks who are not squarely in Mr. Job’s camp.

However, if you take a quick minute to assess Apple’s competition, you will find some striking similarities:

  1. Most of them use chipsets, display, memory etc. from a handful of suppliers – Qualcomm, nVidia, Sony, Toshiba etc.
  2. All devices have relatively similar pricing and depend on the telecom operators to decide on price in markets that support handset subsidies
  3. Some of these devices bear the striking resemblance to the iPhone
  4. As the technology evolution continues, the top spot is enjoyed by a ‘super’ smartphone for 2 -3 months at a time

So where is the differentiation? It is in the industrial design of the product and a thin veneer of usability related ‘value-add’ software, which can be provided over the Operating Systems like Android and Windows Phone 7.  Such usability software like HTC’s Sense UI and Motorola’s Motoblur seem competent additions.

Advancements in technology bring about reduction in component pricing.  Paired with availability of global resources, this has made the mobile device industry look like the computer manufacturing industry of the late nineties – largely involved in assembly work from a myriad of hard-drive, memory, graphics and motherboard manufacturers. Packaging, industrial design and pricing are the only differentiators.  Product refresh cycles have been more accelerated than ever before and consumers have become ever more demanding.

In this industry, the barrier to entry has been significantly lowered. Big names like Dell, Sony-Ericsson, Lenovo have entered the market place with their offerings – some better than others.  With little-known Indian manufacturers like Micromax coming out of $100 android-based smartphones, are Apple and other ‘Tier-1’ manufacturers desperate for differentiation?  Damn right they are!