To administer code coverage for big projects or any project for that matter, we can employ a simple workflow to help enable everyone understand the process. Like any other work flows , this is just an indicator and can be changed to suit any project...
Code Coverage using VSTS
Pre-requisite Build having dll’s/exe along with their corresponding pdb’s VS 2008 Limitations: Code coverage of VSTS is limited to 32 bit. 64 bit dll, during instrumentation will throw error. This implies that instrumentation...
Leveraging existing test automation tools
In an approach to improve the test efficiency in automation, we can think of many areas of improvement. But there can be nothing better than identifying the problem space, like we have done earlier, it is always seen that Test automation tools/frameworks...
Code coverage analysis for API
Code coverage is one (not the only) metric you can use to measure the effectiveness of your test cases. Usually, you’ll use it to find holes in your test model or test coverage. For example, if you run your full suite of tests for an API, and then code...
API Testing best practices
When developing your API tests, keep in mind the following types of test deliverables 1. Test plan Best Practice: Track all your deliverables as work-items in any test management tool of your choice. This provides a great high-level view of what our...
Research in testing
Leveraging research in testing – how can we improve the testing we do to be more efficient and effective? How to make testing more interesting for researchers to work with? What is our pipeline into research? Any takers?...
Testing Types
This can get bigger. But we can restrain it if we know the scope. But these questions are worth finding answers 1. Building automation around integration scenarios 2. Debugging responsibilities – Classical resolution is; because you built the system,...
Risk Management
Its the most important but often most neglected. If we can answer these, then i think we are good to go in any project 1. Informal Risk Management --I care more about its being ineffective than its informality 2. Risk Management --More people need...
Product wide issues
Well these happen in all products or application when the ship time nears 1. Product-wide perf vs. individual teams owning their own perf 2. Product-wide stress and MTTF 3. Test Matrix:...
Automation Challenges
Sometimes these are conveniently ignored. If we can work on getting answers to these, automation engineers( SDET’s) can relax a bit 1. Maintenance nightmares 2. How to simulate eyes/ears 3. Lack of consistency in automation GUID's 4. Non persistent...
Testing Influence
Another big thing in testing that virtually controls the flow 1. Power of Go/No-go ship decisions 2. Design time input – testability concerns 3. Fighting for bugs – i.e. How to effectively argue against "No one is ever going to do that!" type of arguments...
Problems in Metrics
Quality is hard to assess and our measurement systems are ineffective resulting in ship decisions that can negatively impact our quality. We use OCA analysis - What else? Are we testing the right things - What classes of things are we missing that our...
Prioritization of tasks
I think we are majorly influenced by these items when we prioritize our task. The great one said – “ Prioritize what;s on your schedule and not what is not “ & Like one of my friend retorted – if there is nothing on schedule how do you prioritize?...
Estimation & Scheduling
Some difficult items on the estimation and scheduling front 1. Short cycle testing (i.e. Projects working with SCRUM style process). 2. Test having input on scheduling decisions. 3. Accurately estimating test time required....
General Concerns
Some of these concerns could be a more or less like a statement and not necessarily an issue. But some how i have over hauled all the questions that I think need answers for. 1. After several release cycles, test suites can grow large and unmanageable...
SMAF - State Machine based test automation framework
Proposed approach for the harness Problem statement Create a model based intelligent automation harness that would make testing cheaper, faster and better. Approach considerations Address the # 2 of the disadvantages of Model based testing ...
Difficult test items
All things are difficult before they are easy I think, in an effort to understand the philosophies of testing and how best we can get things going, If we can list down the top issues we face and try to find some patterns for them , it would be a good...
Sample Release checklist
[Me]: Guruji, can you please help me with a sample release checklist for any product? [Guruji]: Well, i can help you with only a sample. You may have to moderate it with respect to your project ...
