Tuesday 23 April 2013

Android Architecture



Linux Kernel
Android OS is built on top of the Linux 2.6 Kernel
–Linux Core functionality
•Memory management
•Process management
•Networking
•Security settings
–Hardware drivers

Android’s native libraries
–Libc: c standard lib.
–SSL: Secure Socket Layer
–SGL: 2D image engine
–OpenGL|ES: 3D image engine
–Media Framework: media codecs
–SQLite: Database engine
–WebKit: Kernel of web browser
–FreeType: Bitmap and Vector
–SufraceManager: Compose window manager with off-screen buffering.

Android Runtime
Core Libraries
Provides the functionality of the JAVA Programming Language
Dalvik VM
A type of Java Virtual Machine
Register based (not stack machine like JVM)
Optimization for low memory requirements
Executes .dex (Dalvik-Executable) files instead of .class
DX tool converts classes to .dex format

Application Framework
•The blocks that our applications directly interact with.
•Important blocks:
–Activity Manager: Manages the activity life cycle of applications
–Content Providers: Manage the data sharing between applications
–Telephony Manager: Manages all voice calls. We use telephony manager if we want to access voice calls in our application.
–Location Manager: Location management, using GPS or cell tower
–Resource Manager: Manage the various types of resources we use in our Application

Applications
•This is where our applications are placed.
•Some pre-installed applications:
–SMS client app
–Dialer
–Web browser
–Contact manager
•As developers, we are able to write an app which replaces any existing system app.

History of Android


2001 search service for wireless device
2005
Acquire Android(Andy Rubin: Danger CEO, Development Sidekick of T-Mobile)
Acquire Skia(2D Graphics for mobile device)
Acquire RegWireless(Browser and Email for mobile device)
Move Engineers from PlamSource(Dianne Hackborn, etc…)
2007 Nov 5: Android announced
2007 Nov 12: Android SDK released by OHA
2007 Dec 14: Bug-fix SDK released
2008 Jan 3: Android Developer Challenge I starts accepting submissions
2008 Feb 13: m5-rc15 SDK released
2008 Apr 14: 1788 total submissions for Challenge I
2008 May 12: Top 50 Applications in Challenge I announced
2008 Nov: Android Phone(G1 Phone by HTC/T-mobile)
2008 Nov: Full Source Open
2009 Apr: HTC Magic
2009 July: HTC Hero, Samsung i7500, Android Netbook, Set-top……
2009 Aug: Android Developer Challenge II 

Open Source

Industry
•Software stack open-sourced under Apache 2.0 license
•Source available after first handsets ship
•Anyone will be able to build a system image

Users
•Users have control of their experience
•They control what gets installed
•They choose the defaults

Developer
•Don not need permission to ship an application
•No hidden or privileged framework APIs
•Can integrate, extend and replace existing components                

Friday 19 April 2013

MOBILE APP TESTING CHECKLIST V

Store specific checks

The app cannot download code to be installed without the users consent.

The app can only get new functionality by way of an upgrade through the app store.

After download, an app should remain working. An app cannot turn off after a few days.

An app can’t be a “trail”, “beta”, “demo” or “test” version.

Apple product names should be spelled correctly in the app. (For instance: IPhonez is wrong).

If the app uses the web, it is not done using third party (i.e. non-Apple) browsers.

You cannot mention other app platforms in your app (for instance: “Also available on android!”)

An app cannot use old interfaces, like for instance the iPod click wheel.

Functionality should be in sync with functionality described in store.

The app can’t use the user’s location without permission.

All url’s in the app code should be fully functional

The location services cannot be used to autonomously control of vehicles or for emergency services.

An app cannot use push notifications without user consent.

Push notification can’t send personal information.

The App may not distribute any private information of users (like Player ID) through the game center.

Ad banners must be hidden when there are no ads available.

The app must respect copyright of apple and other parties.

An app can’t restrict the users of the app for instance by location or carrier.

The in app purchase mechanism cannot be used to purchase goods and services used outside the app.

The in app purchase mechanism cannot be used to collect money for charities. This has to be done through SMS.

The in app purchase mechanism cannot be used to buy a raffle or lottery ticket directly from the app.

Apps that encourage the users to use the device in a way that may damage the device will be rejected

An app cannot require user’s personal information (for instance email address) in order for it to function.

An app cannot use location services of the device without asking permission.

An app cannot send spam or introduce viruses, or use other apple platforms like Game Center and Push Notifications to do so.

Push notifications have to be send using the Apple Push Notification (APN) API. This has to be done using an APN ID.

Multitasking functionality of the app can only be used for its intended purposes, i.e. VoIP, Audio playback, location, task completion, local notifications, etc. This means that generally an app can’t run in the background but has to be closed off if it’s not used any more.

The app must have some functionality. For instance, it can’t be just a title page leading to some text. It can’t be just a song, movie or book as there are different platforms for that.

The app can’t use any “non-public API’s”. This means that you can’t use some functions that the distributing platform uses for its own apps. (This can generally be checked best by some sort of automated tool, like http://www.chimpstudios.com/appscanner/)

The app can’t reprogram controls of the device that are not intended for that use. (For instance: using the volume button as a shutter for the camera).

The app should aim at backing up a minimum of information on iCloud. The information in iCloud should be just the user generated information. Information that can be recreated or downloaded should not be backed up.

The app should not access information on the device outside the app without the user’s permission (for instance, copying the address book or getting information from other apps).

The app should not access or write files outside the “Bundle” and “Documents” directory. (because the app can’t read or write data outside the designated container area).

In general, the app has to be decent. So no explicit material in the sense of sex, violence, drugs, alcohol or tobacco. It cannot address a specific ethnic or religious group in a derogatory way.

The app has to be honest. This means that the description of the app has to be correct, and all functionality has to work as described. If an app gives diagnostic information, it has to be reliable. This also means that the genre and category in the  description must be appropriate. The app icons should be consistent and appropriate.

MOBILE APP TESTING CHECKLIST IV

App specific checks

Has the app been tested on different type of devices and different versions of OS?

Stability check: if the app has a list (for instance of pictures) in it, try scrolling through it at high speed.

Stability check: if the app has a list (for instance of pictures) in it, try scrolling to before the first picture or behind the last picture.

Is downloading of the app prevented in case it’s bigger than the OS allows downloading when connected to cellular networks.

Integration: does the app connect correctly to the different social networks (LinkedIn, twitter, facebook, etc).

The app does not interfere with other apps when in background/multitasking mode (using GPS, playing music, etc.).

Can the user print from the app (if applicable) The search option in the app displays relevant results

Verify most common gestures used to control the app.

What happens if you select different options at the same time (undesired multitouch, for example – select two contacts from the phone book at the same time).

App name should be self explanatory

Does the app limit or clean the amount of cached data.

Reloading of data from remote service has been properly designed to prevent performance issues at server-side.

Does the app go to sleep mode when running in the background (prevent battery drain)

MOBILE APP TESTING CHECKLIST III

App UI checks

Use at most one action on the screen that is highlighted as the most likely for the user. (Example: in iOS a blue button represents the default or most likely action).

To keep controls as unobtrusive as possible for instance by fading them out if they are not used for a while.

Make it possible for users to go back to a previous screen for instance by adding a back or cancel button

The main function of the app should be apparent immediately. It should speak for itself.

If the app is stopped at an unexpected time, user data should be saved locally and available at start-up.

Do not use standard buttons for other functions then that they are normally used for

Users should be warned of the consequences of deleting a document Keyboard adjusts to expected input (for instance numbers/letters when expected).

Are inactive buttons clearly distinguished from active buttons?

Minimize user actions by using a picker or a table view where users can select a certain choice over a data entry field where users have to type a choice

In an app, the user should not be able to store files locally, outside the app sandbox.

In an app, the user should not be exposed to the permissions of a specific file

Tapable elements should be about 7x7 mm in size, using the pixel density of the target device you can calculate the amount of pixels (chapter documentation contains a link to different devices compared).

Do not redefine gestures in your app that have a standard meaning (example:swiping from top to bottom enables the notification center)

Requirement to login is delayed in the app as long as possible

In case of ‘live’ filtering of data while the user enters his search query, verify the performance.

The appearance of buttons that perform standard actions are not altered in the app (for instance: refresh, organize,trash, Reply, back, etc.)

If there is a long list of data to scroll trough, provide a search option above the list.

If performance is slow, indicate a progress status icon (“Loading…”), preferably with specific message.

The app should respond to all changes in device orientation, as per the design

MOBILE APP TESTING CHECKLIST II


Device specific checks

Can the app be installed on the device?

Does the app behave as designed/desired if there is an incoming call?

Does the app behave as designed/desired if there is an incoming SMS?

Does the app behave as designed/desired if the device is tilted?

Does the app behave as designed/desired if the device is shaken?

Does the app behave as designed/desired if a local message is coming from another app (think of: calendar reminders, to-do task etc.).

Does the app behave as designed/desired if a push message is coming from another app (think of: twitter mentions, whats app message, word feud invitation, etc).

Does the app behave as designed/desired if the charger is connected?

Does the app behave as designed/desired if the charger is disconnected?

Does the app behave as designed/desired if the device goes to sleeping mode

Does the app behave as designed/desired if the device resumes from sleeping mode

Does the app behave as designed/desired if the device resumes from lock screen?

Does the app interact with the GPS sensor correctly (switch on/off, retrieve GPS data)?

Is the functionality of all the buttons or keys on the device defined for this app?

Verify that buttons or keys which have no defined function have no unexpected behaviour on the app when activating.

Does the app behave as designed/desired if the “Battery low” message is pushed

Does the app behave as designed/desired if the sound on the device is turned off?

Does the app behave as designed/desired if the device is in airplane mode?

Can the app be de-installed from the device?

Does the application function as expected after re-installation?

Can the app be found in the app store?

Can the app switch to different apps on the device through multitasking as designed/desired?

Are all touch screen positions (buttons) working when a screen protector is used.

In case there’s a true “back” button available on the device does the “back” button take the user to the previous screen?

In case there’s a true “menu” button available on the device, does the menu button show the app’s menu?

In case there’s a true “home” button available on the device, does the home button get the user back to the home screen of the device?

In case there’s a true “search” button available on the device, does this get the user to some form of search within the app?

MOBILE APP TESTING CHECKLIST

Checklist comprises of below mentioned categories:


  • Network specific checks
  • Device specific checks.
  • App UI checks.
  • App specific checks. (These are related with functionality that is frequently used in an app.)
  • Store specific checks

Network Specific Checks

Does the app behave as per specification if connected to the internet via Wi-Fi?

Does the app behave as per specification if connected to the internet via 2G?

Does the app behave as per specification if connected to the internet via 3G?

How the app behave if out of network reach?

How the app behave if it is in low network reach?

Does the app behave as per specification if navigating through application screens and Airplane mode is activated.

Does the app behave as per specification if while playing media content, Airplane mode is activated.

Does the app behave as per specification if while initiating a call from Device, Airplane mode is activated.

Does the app behave as per specification if while sending a SMS from Device, Airplane mode is activated.

Does the app resume working when it gets back into network reach from outside reach of the network?

Does the update transactions are processed correctly after re-establishing connection. 

Does the app still work correctly when tethering or otherwise connected to another device

What is the behaviour of the app if network switches between  (Wi-Fi, 3G, 2G)

Does the app use standard network ports (Mail: 25, 143, 465, 993 or 995 HTTP: 80 or 443 SFTP: 22) to connect to remote services, ( as some providers block certain ports.)

Thursday 18 April 2013

Steps for Connecting to the Remote Servers



 Pre-requisite:

1. Download the PuTTYgen and PuTTY from the URL: http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
2. Generate the SSH Key Pair (Public Key & Private Key) using PuTTYgen by following the steps mentioned at URL:
3. Save the Public Key and Private Key. Public key needs to be on the server that you want to connect to and Private key is to be used on the system that you are using to connect to remote server.
Settings to be done in Putty: 

1. Sessions Settings
Host Name (or IP Address): builduser@build4.appcentral.com
Port: 22
Connection Type: SSH
2. Then goto Connection >> SSH >> Tunnels
Add new forwarded port:
Source port: 3306
Destination: localhost:3306
Click on the "Local" radio button.
Click "Add"
3. Then goto Connection >> SSH >> Auth
Browse the Private Key in the field: Private Key file for Authentication
4. Go back to Session settings, type a name for Saved session and click "Save". Then click "Open"

From Next time just need to select the saved session and then click on Load then Open OR simply double click on the session name to get connected.

Getting the Logs on the server: 
  1. Navigate to directory: /opt/ac/tomcat/logs/catalina.out
  2. Write a command : ‘tail –n 1000 catalina.out >> Logs’ – (This command will get you the last 1000 lines from Catalina.out and will copy into a file named Logs and save it to the current directory)
  3. Now use command cat <filename> to read the contents of the files. In this case it should be ‘cat Log’
 Other commands that can help : 
  1. To move one folder back: cd ..
  2. To see the files and folders in a particular directory: ls OR ls –l
  3. To get inside a folder: cd <foldername>
  4. To remove the files/folders: rm <file/foldername> 
Using WinSCP for fetching the files from Remote (Linux server) server to your local system. 
  1. Download and Install WinSCP using the URL: http://winscp.en.softonic.com/universaldownloader-launch
  2. Provide the Host Name as Provided in Putty configuration and Upload the Private Key (the one used in Putty configuration)
  3. Then Connect. 

Wednesday 17 April 2013

How to measure and analyze the testing efficiency?


Measurements or Metrics or Stats are the common terms you would hear in every management meeting. Some basic numbers that reflect speed of testing, coverage of testing, efficiency of testing are described here. If all these indicators move up, we can definitely be confident that the testing efficiency is getting better.

Test planning rate (TPR). TPR = Total number of test cases planned / total person-hours spent on planning. This number indicates how fast the testing team thinks, articulates the tests and documents the tests.

Bug Dispute Rate (BDR). BDR = Number of bugs rejected by development team / Number of total bugs posted by testing team. A high number here leads to unwanted arguments between the two teams.

Test execution rate (TER). TER = Total number of test cases executed / total person-hours spent on execution. This indicates the speed of testers in executing the same.

Planning Miss (PM).  PM = Number of adhoc test cases that are framed at the time of execution / Number of test cases planned before execution. This indicates, whether the testers are able to plan the tests based on the documentation and understanding levels. This number must be as less as possible, but it is very difficult to achieve zero level in this.

Requirements coverage (RC). Ideal goal is 100% coverage. But it is very tough to say how many test cases will cover 100% of requirements. But there is a simple range you mus assume. If we test each requirement in just 2 different ways - 1 positive and 1 negative, we need 2N number of test cases, where N is the number of distinct requirements. On an average, most of the commercial app requirements can be done with 8N test cases. So, the chances of achieving 100% coverage is high if you try to test every requirement in 8 different ways. Not all requirements may need an eight-way approach.

There is a set of metrics that reflect the efficiency of the development team, based on the bugs found by the testing team. Those metrics do not really reflect the efficiency of the testing team; but without testing team, those metrics cannot be calculated. Here are a few of those.

Bug Fix Rate (BFR). BFR = Total number of hours spent on fixing bugs / total number of bugs fixed by dev team. This indicates the speed of developers  in fixing the bugs.

Bug Bounce Chart (BBC). BBC is not just a number, but a line chart. On the X axis, we need to plot the build numbers in sequence. Y axis contains how many New+ReOpen bugs are found in each build. Ideally this graph must keep dropping towards zero, as quickly as possible. But if we see a swinging pattern, like sinusoidal wave, it indicates, new bugs are getting injected build over build, due to regression effects. After code-freeze, product companies must keep a keen watch on this chart.

Number of re-opened bugs. This absolute number is an indicator of how many potential bad-fixes or regression effects are injected into the application, by the development team. Ideal goal is zero for this.

Ten Software Testing Myths


10. The tester’s task is easy: he should merely write and execute the test cases by translating requirements to test cases. Additionally log some bugs.

9. Every test case is documented. Otherwise, how on earth can we expect to do regression testing and in general repeat testing?

8. Test case Reviews are a one-time effort. All you have to do is take an artifact after it is completed, and verify that it is correct. Test case reviews, for example, should merely verify that *all* requirements are covered by test cases and EVERY REQUIREMENT is COVERED by AT LEAST ONE TEST CASE.

7. Software Testing should be like manufacturing. Each of us is a robot in an assembly line. Given a certain input, we should be able to come up automatically with the right output. Execute a set of test cases (should execute 100 test cases a day) and report pass/fail status.

6. Software Testing has nothing to do with creativity. Creativity – what? The only part which requires creativity is designing your assembly line of test case design. From that point on, everyone should just be obedient.

5. Creativity and discipline cannot live together. Creativity equals chaos. [This one remains unchanged from original list of software development myths]

4. The answer to every challenge we face in the software industry lies in defining a process. That process defines the assembly line without which we are doomed to work in a constant state of chaos. [BIG ONE …This one remains unchanged from original list of software development myths]

3. Processes have nothing to do with people. You are merely defining inputs and outputs for different parts of your machine.

2. If a process is not 100% repeatable, it is not a process. Letting people adapt the process and do “whatever they want” is just going back to chaos again.

1. Quality is all about serving the customer. Whatever the customer wants, he should get. Things that don’t concern your customer should not be of interest to you.

Tuesday 16 April 2013

Software Testing Terminology


Error - The difference between a computed, observed, or measured value or condition and the true, specified, or theoretically correct value or condition.

Fault - An incorrect step, process, or data definition in a computer program.

Debug - To detect, locate, and correct faults in a computer program.

Failure - The inability of a system or component to perform its required functions within specified performance requirements. It is manifested as a fault.

Testing - The process of analyzing a software item to detect the differences between existing and required conditions (that is, bugs) and to evaluate the features of the software items.

Static analysis - The process of evaluating a system or component based on its form, structure, content, or documentation.

Dynamic analysis - The process of evaluating a system or component based on its behavior during execution.

Correctness - The degree to which a system or component is free from faults in its specification, design, and implementation. The degree to which software, documentation, or other items meet specified requirements. The degree to which software, documentation, or other items meet user needs and expectations, whether specified or not.

Verification - The process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. Formal proof of program correctness.

Validation - The process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements.

Tip to create workflow in JIRA quickly


To my colleagues who want to create quickly new workflow and don't want to touch xml \ property files.

On Administrator \ Workflows

If you don't have default workflow then create default Workflow with default JIRA "workflow" - it will show status as "active" so we cannot update its scheme \ workflow steps.
Copy this workflow -- it has status as "Inactive" so we can update workflow scheme.
Go to link "steps" and add new status \ transitions for new steps as required in your workflow.

On Administrator \ Workflow Schemes

Create new "Workflow Scheme" then click on "Workflows" link operation of new workflow just created.
Assign workflow to scheme,... Issue type "Bug" and select our workflow from the list.
Assign other workflows for issue types : Task, New Feature or Improvement with default workflow ( named JIRA ) or your workflow.

Now our workflow scheme is ready to associate with our project.
Go to your project admin panel and set workflow scheme there.

What are advantages or disadvantages of automated testing?


Following are the advantages for using automated tools: 
  • Fast: Human be slow but Automated Testing Tool runs tests significantly faster than human users.
  • Reliable: Tests perform precisely the same operations each time they are run, thereby eliminating human error.
  • Repeatable: You can test how the software reacts under repeated execution of the same operations.
  • Programmable: You can program sophisticated tests that bring out hidden information from the application.
  • Reusability: You can reuse the Automated Test scripts, user defined functions etc. 

Automated tools have following disadvantages: 
  • Usability Testing: Usability of software can not be automated so automated tools cannot rate the usability of an application.
  • Cost:  Usually these are licensed so automated tools are quite expensive
  • Programming knowledge required: To customize test scripts according to test requirement. Test maintenance is costly: In case of playback methods. Even though a minor change occurs in the GUI, the test script has to be rerecorded or replaced by a new test script.

What is a test cases review?

Software Testing plays a vital role in ensuring the quality of a Software product. Test cases are the key tools for testing. Test cases review has to be thorough in order to ensure that effective and adequate testing is done

What is the importance of test cases review?
  • Test cases are written with the intent to detect the defects
  • Understanding of the requirement is correct
  • Impact areas are identified and brought under test
  • Test data is correct and represent every possible class of the domain
  • Positive and negative scenarios are covered
  • Expected behavior is documented correctly
  • Test coverage is adequate
What is the methodology of test cases review?

Typically, reviews should be done during each phase in the Testing Life cycle
The phases involved in Software testing lifecycle are

Requirements understanding --During the Requirements Understanding phase, review of requirements is an activity that should be undertaken with utmost care and such review should be done systematically to ensure the clarity, correctness and testability of the requirements.
Test preparation--During the Test preparation phase, after the test scenarios are identified and test conditions and cases are built for each scenario, it is advisable to do a thorough and detailed review. Checklist for Test cases review during the Test Preparation Phase
Test Execution--During the test execution phase, doing a review after the cases are executed is very important .
Test Reporting--During the Test Reporting phase, it would help to ensure that all the required documents are prepared, metrics are collated and all the project specific formalities are completed.

What are the most common test case review defects?
  • Incomplete Test Cases
  • Missing negative test cases
  • No Test Data
  • Inappropriate/Incorrect Test data
  • Incorrect Expected behavior
  • Grammatical errors
  • Typos
  • Inconsistent tense/voice
  • Incomplete results/number of test runs
  • Defect details not updated
  • Changes to requirements not updated in Test case

Monday 15 April 2013

Difference between Severity and Priority?

Severity
Priority
In simple words, severity depends on the harshness of the bug.
In simple words, priority depends on the urgency with which the bug needs to be fixed.
It is an internal characteristic of the particular bug. Examples of High severity bugs include the application fails to start, the application crashes or causes data loss to the user.
It is an external (that is based on someone's judgment) characteristic of the bug.
Examples of high priority bugs are the application does not allow any user to log in, a particular functionality is not working or the client logo is incorrect. As you can see in the above example, a high priority bug can have a high severity, a medium severity or a low severity.
Its value is based more on the needs of the end-users.
Its value is based more on the needs of the business.
Its value takes only the particular bug into account. For example, the bug may be in an obscure area of the application but still have a high severity.
Its value depends on a number of factors (e.g. the likelihood of the bug occurring, the severity of the bug and the priorities of other open bugs).
Its value is (usually) set by the bug reporter.
Its value is initially set up by the bug reporter. However, the value can be changed by someone else (e.g. the management or developer) based on their discretion.
Its value is objective and therefore less likely to change.
Its value is subjective (based on judgment). The value can change over a period of time depending on the change in the project situation.
A high severity bug may be marked for a fix immediately or later.
A high priority bug is marked for a fix immediately.
The team usually needs only a handful of values (e.g. Showstopper, High, Medium and Low) to specify severity.
In practice, new values may be designed (typically by the management) on a fairly constant basis. This may happen if there are too many high priority defects. Instead of a single High value, new values may be designed such as Fix by the end of the day, Fix in next build and Fix in the next release.

How to do exhaustive testing?


Testing a software application (except maybe a very simple program a few lines long) may well be an impossible task due to large number of:

1. All possible inputs
2. All possible input validations
3. All possible logic paths within the application
4. All possible outputs
5. All possible sequences of operations
6. All possible sequences of workflows
7. All possible speeds of execution And the above for just with a single user
8. All combinations of types of users
9. All possible number of users
10. All possible lengths of time each user may operate the application
And so on (we have not even touched the types of test environments on which the tests could be run).

However, it is possible to exhaustively execute your test suite using the following tips:

1. Your test suite should have test cases covering each documented requirement. Here my assumption is that each requirement is documented clearly.
2. The test cases should be specific, concise and efficient. Each test case should have clear and unambiguous steps and expected results.
3. The configuration data, input test data and output test data should be clearly specified.
4. You should have a clean and stable test environment in which to execute your test suite.
5. In a perfectly working application, it should be possible to execute each test case in the suite.
6.. Each confirmed bug (found during testing or found by the client) should result in another test case written or an existing test case updated.
7. Important: You should not assume the correctness and completeness of your test suite by yourself. Review of such test suite by peers, business people, managers, clients and users may provide you valuable inputs to correct your test suite.
8. Discipline in maintaining your test suite and executing it would go a long way in preventing bugs leaked to the clients/ users of your application.

How to find more or good bugs


Tip 1. Review the application's requirements often. Sometimes there are no or partial test cases exist for certain requirements. You may find bugs when you test for these requirements. Be quick and you may find bugs immediately after a requirement change has been first implemented.
Tip 2. It is possible that you have created positive test cases only. If so, then create negative test cases for each requirement as well and test them.
Tip 3. Execute each test case with a variety of test data (for example by the boundary-value analysis technique or pair-wise testing technique).
Tip 4. Test the interface of your application with external systems carefully. This is where a number of bugs may exist.
Tip 5. Application settings (this is one area that may be rarely tested in fear that the application may stop working correctly or stop working altogether).
Tip 6. Repeat your tests using other supported client configurations (other CPU/RAM/operating systems/screen resolutions/browsers etc.)
Tip 7. Look at the previous bug reports against the same application or similar applications. See if you can test your application using the ideas or information contained in the previous bug reports.
Tip 8. Do not ignore the UI bugs. If they would inconvenience a user, they should be reported and fixed.
Tip 9. Create a list of items that you would like to test if you had the time. When you test your application, you may find yourself taking mental notes of things that you would like to test later.
Tip 10. In case you are working as part of a testing/ QA team, do not restrict yourself to the areas that you are supposed to test. You may find bugs in the other areas.
Tip 11. Instead of rushing through your tests at top speed, slow down . You would give yourself time to think more clearly. You would start to observe things about your application that you did not observe before.
Tip 12. Finally, take pride in the bugs reported by you. Feel free to mention an interesting bug that you found to a team member.

Saturday 13 April 2013

Cookie Testing


Major scenarios for cookies testing of a website. Multiple test cases can be generated from these scenarios by performing various combinations.
  1. Check if the application is writing cookies properly or not.
  2. Test to make sure that no personal or sensitive data is stored in the cookie. If it is there in cookies, it should be in encrypted format.
  3. If the application under test is a public website, there should not be overuse of cookies. It may result in loss of website traffic if browser is prompting for cookies more often.
  4. Close all browsers, delete all previously written cookies and disable the cookies from your browser settings. Navigate or use that part of web site which use cookies. It should display appropriate messages like "For smooth functioning of this site please enable cookies on your browser."
  5. Set browser options to prompt whenever cookie is being stored / saved in your system. Navigate or use that part of web site which use cookies. It will prompt and ask if you want to accept or reject the cookie. Application under test should display an appropriate message if you reject the cookies. Also, check that if pages are getting crashed or data is getting corrupted.
  6. Close all browsers windows and manually delete all cookies. Navigate various web pages and check and see if these web pages show unexpected behavior.
  7. Edit few cookies manually in notepad or some other editor. Make modifications like alter the cookie content, name of the cookie, change expiry date etc. Now, test the site functionality. Corrupted cookies should not allow to read the data inside it.
  8. Cookies written by one web site should not be accessible by other website.
  9. If you are testing an online shopping portal, Check if reaching to your final order summary page deletes the cookie of previous page of shopping cart properly and no invalid action or purchase got executed from same logged in user.
  10. Check if the application under test is writing the cookies properly on different browsers as intended and site works properly using these cookies. This test can be done on browsers like different versions of internet explorer, Mozilla Firefox, Netscape, Opera etc.
  11. If the application under test is using cookies to maintain the logging state for users. Check if some id is being displayed in the address bar. Now, change the id & press enter. It should display an access denied message and and you should not be able to see other user's account.

Tuesday 9 April 2013

Top 10 Negative Test Cases



1. Embedded Single Quote - Most SQL based DB’s have issues when users store information that contain a single quote (e.g. Dave's car). For each screen that accepts alphanumeric data entry, try entering text that contains one or more single quotes.

2. Required Data Entry - Functional specification should clearly indicate fields that require data entry on screens. Test each field on the screen that has been indicated as being required to ensure it asks you to enter data in the field.

3. Field Type Test - Functional specification should clearly indicate fields that require specific data entry requirements (date fields, numeric fields, phone numbers, zip codes, etc). Test each field on the screen that has been indicated as having special types to ensure it forces you to enter data in the correct format based on the field type (numeric fields should not allow alphabetic or special characters, date fields should require a valid date, etc)

4. Field Size Test - Functional specification should clearly indicate the number of characters you can enter into a field (for example, the first name must be 50 or less characters). Write test cases to ensure that you can only enter the specified number of characters. Preventing the user from entering more characters than is allowed is more elegant than giving an error message after they have already entered too many characters.

5. Numeric Bounds Test - For numeric fields, it is important to test for lower and upper bounds. For example, if you are calculating interest charged to an account, you would never have a negative interest amount applied to an account that earns interest, therefore, you should try testing it with a negative number..

6. Numeric Limits Test - Most database systems and programming languages allow numeric items to be identified as integers or long integers. Normally, an integer has a range of -32,767 to 32,767 and long integers can range from -2,147,483,648 to 2,147,483,647. For numeric data entry that do not have specified bounds limits, work with these limits to ensure that it does not get an numeric overflow error.

7. Date Bounds Test - For date fields, it is important to test for lower and upper bounds. For example, if you are checking a birth date field, it is probably a good bet that the person's birth date is no older than 150 years ago. Likewise, their birth date should not be a date in the future.

8. Date Validity - For date fields, it is important to ensure that invalid dates are not allowed (04/31/2007 is an invalid date). Your test cases should also check for leap years (every 4th and 400th year is a leap year).

9. Web Session Testing - Many web applications rely on the browser session to keep track of the person logged in, settings for the application, etc. Most screens in a web application are not designed to be launched without first logging in. Create test cases to launch web pages within the application without first logging in. The web application should ensure it has a valid logged in session before rendering pages within the application.

10. Performance Changes - As you release new versions of your product, you should have a set of performance tests that you run that identify the speed of your screens (screens that list information, screens that add/update/delete data, etc). Your test suite should include test cases that compare the prior release performance statistics to the current release. This can aid in identifying potential performance problems that will be manifested with code changes to the current release.

Testing GUI



To test any Windows based application following points are to be considered:

- It is essential to have GUI consistency within the Application.
- Should be alike in look and feel as per any other standard Window software.
- Should have standard set of keys implemented for the software.
- Should have clean and neat exit.

While testing any Windows based application the testing can be broadly categorized into following compartments. They are:
- Standardization Testing.
- GUI Testing.
- Validation Testing.
- Functionality Testing.

Standardization Testing
This compartment mainly emphasizes on the standardization part of the application. Standardization means that the application being developed should have standard look and feel like any other window application. The general guidelines are as follows:
1. The application should have first "About Application" screen displayed.
2. Most of the screens/ dialog box (as on context) should have Minimize, Restore and Close clicks.
3. Proper icon should be attributed to the application
4. All screens/ dialog box should have a proper caption as per the context used.
5. The application should be seen in the Windows Task Bar as well as status bar.

GUI Testing
This compartment mainly emphasizes on the GUI - Graphics User Interface aspect of the Application. It is not concrete that once GUI guidelines are set that can be followed blindly. GUI standards may vary from company to company and also from application to application. But still one can set general guidelines to have an overall idea on how to start GUI testing. These guidelines apply for every screen/ dialog box of the application. General guidelines are:
1. All the dialog box should have a consistent look through out the Application system. For e.g.- If the heading within a dialog box is blue then for each dialog box the heading should be of this color.
2. Every field on the screen should have an associated Label.
3. Every screen should have an equivalent OK and cancel button.
4. The color combination used should be appealing.
5. Every field in the dialog box should have a Short Cut Key support. For e.g.- User Name
6. Tab order should be normally set horizontally for the fields. In some case as per the case the Tab Order can be set vertically.
7. Mandatory fields should have * (RED ASTERIK) marked to indicate that they are mandatory fields.
8. Default key <Enter> should be set as OK for the dialog box.
9. Default key <Esc> should be set as Cancel for the dialog box.

Validation Testing
This compartment mainly emphasizes on the Validation aspect of the Application. Validation testing mainly depends on the fields set in the dialog box and the functions it has to perform. But still there are certain common rules that can be applied. General guidelines are:
1. For text box fields where value entered has to be numeric check following:
¨ It should accept numbers only and not alphabets.
¨ If field usage is such that for e.g., To accept
Total number of days
Telephone number
Zip code etc.
then it should not accept 0 and negative values.
2. For text box fields where value entered has to be alpha-numeric check following:
¨ It should accept alphabets and numbers only.
¨ If field usage is such that for e.g., accepting
First Name
Middle Name
Last Name
City
Country etc.
then field value should start with an alphabet only.
¨ Depending on the condition this fields may accept special characters like -, _, . etc.
3. If the field is a combo box then it has to be checked for following points:
¨ Check the combo box has drop down values in it, it is not empty.
¨ Drop down values should be alphabetically sorted. This might change as per requirement but as standard practices it should be alphabetically sorted. For e.g. to select data type from the list it will be as follows:
Date
Integer
String
Text, etc.
¨ Selection of any drop down value is displayed on closing and opening the same dialog box.
¨ By default some value like "Select Value" or "_______" string is displayed. This is because User comes to know that value is to be selected for this field. Avoid displaying the first default value in the list.
4. If the field is a list box then it has to be checked for following points:
¨ Check the list box has values in it, it is not empty.
¨ List box values should be alphabetically sorted and displayed. This might change as per requirement but as standard practices it should be alphabetically sorted.
¨ Selection of any list box value should put a check before the value and should display the correct value(s) selected on closing and opening of the same dialog box.
¨ If the list box supports multiple selection then check whether multiple values can be selected.
5. If the field is a list of radio button then it has to be checked for following points:
¨ Check whether as per requirements all the values are listed. For e.g. to select date format. Possible values displayed will be as follows:
mm/dd/yyyy
dd/mm/yyyy
mm/dd/yy
dd/mm/yy
yyyy/mm/dd etc.
¨ Same selected value should be displayed on closing and opening of the same dialog box.
6. Data Controls are to be tested as part of functionality testing.

Functionality Testing
This compartment mainly emphasizes on the Functionality aspect of the Application. The first step to test the Functionality aspect of the Application is to check whether all the requirements are covered in the software. The actual functionality testing totally depends from software to software. Still one can frame general guidelines. General guidelines are:
1. Check the functionality is covered as per Requirement specifications or Functional specifications developed for the software.
2. Within a dialog box identify the dependent fields. Depending on the dependency check the enabling and disabling of the fields. For e.g.: to create Contact addresses in any application. To create contact addresses user should be able to add, delete and modify the information. Contact Addresses will contain information like, First Name, Last Name, Address1, Address2, City, State, Country, Zip, Phone, etc., any other information may also be added.
This form will have the required fields and in addition to that will have Add, Delete and Update buttons. The functionality of the buttons is as follows:
¨ Initially only Add button will be enabled. Delete, Update buttons will be disabled. This is because initially there is no data available and unless one adds one cannot delete or update. In short, unless there is a single valid record available it is not possible to update or delete.
¨ Only on selecting a record from the list Delete and Update buttons are enabled and Add button is disabled. By default No records will be selected.
¨ Delete and Update should always give confirmation message before actually performing the operation.
¨ Delete operation should not show the deleted item in the list.