1 of 40

A Comparison of Individual and Group-Based Effort Estimation Techniques�-�Ghufran Ahmad (FAST School of Computing)�Dr. Ali Afzal Malik (FAST School of Computing)��

2 of 40

Presentation Agenda�

AGENDA

LITERATURE REVIEW

EXPERIMENT RESULTS

RESULTS ANALYSIS

INTRODUCTION

RESEARCH QUESTIONS

PROBLEM?

EFFORT EST. TECHNIQUES

CONCLUSION & FUTURE WORK

3 of 40

INTRODUCTION

4 of 40

Introduction

Project Management

Essential Skill
Complex Nature of Projects
Rapidly Changing Requirements

Estimation

Size of Project
Effort and Duration
Cost

Risk Management

Risk Identification
Mitigation Plans made

As organizations move towards agility, it is the responsibility of project managers to provide quality products within reasonable time duration and cost. Hence, PM has become an essential skill for managers to match the industry advancements. If we look into our software industry, usually projects are complex in nature and are being completed under such an environment where business and technological nature keeps changing during a project.
One of the major tasks for software project managers is estimating the effort, time, and cost required by the various project activities. These estimates are essential for project planning and scheduling. Inaccurate estimates may lead to schedule slippage, budget overruns, and, ultimately, project failure.
Last but not the least, after estimation risk management is essential skill required for Project Managers. Risk planning and management is another major factor in the life cycle of a project. Here all type of risks associated with the project are kept under consideration and plans are made to avoid these risks well before time.

5 of 40

EFFORT ESTIMATION�TECHNIQUES

6 of 40

Background Information�

Single Point Estimation

Three Point Estimation - N

Three Point Estimation - P

Wideband Delphi Estimation

Expert Judgment Based
Fixed Estimate of Time
Best Guess/ Best Estimate

Expert Judgment Based
Optimistic, Pessimistic and Most Likely Estimates
Equal Weightage for Each Estimate

Expert Judgment Based
Optimistic, Pessimistic, and Most Likely Estimates
Most Likely gets 4x more Weightage

Group-Based Estimation
More Communication and Group Interaction
More Reliable and Accurate Estimates

Single Point Estimation (SPE) is a fixed estimate of time, such as two person-days or one person-month. This method is a widely used expert judgment based estimation technique. In this method each expert based on his/her prior experience and understanding based on requirements gives the estimated effort which is to serve as a "Best Guess" or "Best Estimate".
For Three Point Estimation, Each expert, after getting a sound understanding of specified requirements, gives three estimates against each user story/module. First guess will be the pessimistic one assuming everything goes bad, the other will be optimistic assuming everything goes in good order and the last one will be the most likely.
We’ll be looking into 2 different variations of TPE , normal distribution where each estimate will hold equal weightage and other is PERT distribution where ‘most likely’ estimate will get 4x weight of optimistic and pessimistic one.
WDE technique is a group-based software effort estimation technique which require group consensus, it promotes more communication and interaction between the group members as compares to earlier discussed individual estimation techniques. It gives consensus based estimates which are generally more reliable and accurate when compared to individual estimates.

7 of 40

Wideband Delphi Estimation

Review Results

Assemble Estimates

Estimation Sessions

Individual Preparation

Kickoff Meeting

Team Selection

1- Team selection: Moderator, is responsible for gathering a team involving customer representatives and technical team members.

2- Kickoff meeting: As the initial meeting, each participant except the moderator try to identify work products and discuss the relevant expectations.

3- Individual preparation: Every single participant identify the effort required for each task identified in WBS and fills in the form provided by the moderator.

4- Estimation session: Moderator collects the individual estimation forms and announces the estimates provided by participants then, participant with worst and best estimates are asked to provide rationale for their estimates. Each participant can re-visit his/her estimates after listening to the rationale behind worst and best estimates in the next iteration

5- Assemble tasks: Moderator assembles the individual estimation forms and fills the gathered information related to estimate against WBS into a single form, called summary sheet and achieve consensus by using graphs for each round of discussions.

6- Review results: As the final step of the WDE process, the moderator alongside different team members analyze the general estimates to check whether or not it seems OK and furthermore, they talk about the improvement of the effort and cost saving strategy.

8 of 40

PROBLEM �?

9 of 40

Problem

Improving Estimation Accuracy

Long Time Debate

Individual vs Wideband Delphi Estimation Technique

Direct Comparison Missing

Improve Wideband Delphi Estimates

Very Little Work Done

10 of 40

RESEARCH QUESTIONS

11 of 40

Research Questions

RQ1: Which of the individual estimation techniques provides the most accurate estimates?

RQ2: Is the group-based WDE more accurate than the individual expert judgment-based estimation techniques?

RQ3: Does using WWDE improve the accuracy of estimates obtained using the original WDE?

12 of 40

PROPOSED�VARIANT

13 of 40

Weighted Wideband� Delphi Estimation

Review Weighted Results

Assemble Estimates+ Ratings

Estimation Sessions + Experts Ratings

Individual Preparation

Kickoff Meeting

Team Selection

14 of 40

LITERATURE�REVIEW

15 of 40

S#	Title	Author	Year	Findings	Limitations
1	Reliability and Accuracy of Estimation Process - Wideband Delphi vs Wisdom of Crowd.	Marek Grzegorz Stochel	2011	The goal was to identify weaknesses and limitations in the estimation practices based on Wideband Delphi method and to propose an alternative solution.	Both estimation techniques are vulnerable to the same common factors which have either psychological or empirical roots.
2	A Case Study Research on Software Cost Estimation Using Experts' Estimates, Wideband Delphi, and Planning Poker Technique.	Gandoma ni T. J., Wei K. T. & Binhamid A. K.	2014	Study showed that applying Planning Poker & Wideband Delphi technique to companies already using expert opinions for estimation increased the accuracy of estimation.	Comparing Wideband Delphi and Planning Poker on the basis of two case studies was not enough to state opinions.
3	Enhancing Delphi Method with Algorithmic Estimates for Software Effort Estimation: An Experimental Study.	Tharwon Arnuphap trairong	2021	Combination of Wideband Delphi and algorithmic estimates leads to slightly better estimates which shows a promising technique for future.	Only one group of 5 experts and less number of software projects were tested which is not enough to draw conclusions.

Literature Review �

16 of 40

RESEARCH METHODOLOGY

17 of 40

Research Methodology

PREPARATION

Projects Data Collection
Expert Selection

EXECUTION

Expert’s Training
Estimation Sessions

COMPARATIVE ANALYSIS

Individual Techniques
Group-Based Techniques
All Techniques

Preparation : This is the very first step of our research, First step in the preparation phase was to design a data collection form, which is further used to gather the already completed project’s information. Once all the data points are collected, the next step in study is to select software industry experts meeting specific criteria. Since experts play an important role in this study, the selection of software experts is a crucial step. People having a minimum of 7-8 years of experience in the software industry along with relevant project management experience were selected for estimation purposes
Execution : Once the preparation phase is completed, the next phase in research is the execution phase, After each group of experts is formed, first all groups will have an introductory session, where they will be explained what all techniques are about and how each session will be conducted. Then, people giving estimates in groups will have an additional session about the detailed procedure of WDE technique and what are the crucial steps to get to final estimates. Then as the last step of this stage will be actual estimation sessions where all selected experts go through the estimation techniques processes for each data point.
Comparative Analysis : Last stage after estimation sessions will be to analyze the gathered estimates through each estimation technique.

18 of 40

Individual Estimation Process

Single Point Estimation Session

User Story
Expert Analysis
Best Estimate Provided

Three Point Estimation Session

User Story
Expert Analysis
Optimistic, Most Likely and Pessimistic Estimates

19 of 40

Group-Based Estimation Process

Requirements

User Story

Analysis
Rating Form
Estimates Filled

Estimation Sessions

Group Discussions
Re-analyze Estimates
Converge Estimates

WDE Estimates

Apply Weights
Ratings
Final Weighted Estimates

Weighted WDE Estimates

20 of 40

RESULT�PREPARATION

21 of 40

Projects’ Information�

Project Name	Platform	Domain
P1	Mobile-Android	Utility
P2	Mobile-Android	Maps & Tracking
P3	Mobile-Android	Utility
P4	Mobile-iOS	Agriculture & Community
P5	WEB	NGO
P6	WEB	Shopping
P7	WEB	Food Restaurant
P8	WEB	Events & Management

22 of 40

Expert’s Selection�

List of Expert’s Participated

23 of 40

EXECUTION�RESULTS

24 of 40

SPE Data�

Project(s)	Actual (PD’s)	Estimated (PD’s)	MMRE	Within 10%?
P1	13	10	0.23	0
P2	17.5	17.7	0.01	1
P3	11.76	10.85	0.08	1
P4	26.13	22.7	0.13	0
P5	10.38	12.6	0.21	0
P6	71.63	77.4	0.08	1
P7	21.63	18.2	0.16	0
P8	29.25	27.8	0.05	1

25 of 40

TPE (N) Data�

Project(s)	Actual (PD’s)	Estimated (PD’s)	MMRE	Within 10%?
P1	13	11.5	0.12	0
P2	17.5	18.62	0.06	1
P3	11.76	13.05	0.11	0
P4	26.13	23	0.12	0
P5	10.38	13.05	0.25	0
P6	71.63	87.85	0.23	0
P7	21.63	20.23	0.06	1
P8	29.25	26.45	0.10	1

26 of 40

TPE (P) Data�

Project(s)	Actual (PD’s)	Estimated (PD’s)	MMRE	Within 10%?
P1	13	11.5	0.08	1
P2	17.5	18.62	0.07	1
P3	11.76	13.05	0.09	1
P4	26.13	23	0.12	0
P5	10.38	13.05	0.28	0
P6	71.63	87.85	0.22	0
P7	21.63	20.23	0.07	1
P8	29.25	26.45	0.09	1

27 of 40

WDE Data�

Project(s)	Actual (PD’s)	Estimated (PD’s)	MMRE	Within 10%?
P1	13	13.3	0.02	1
P2	17.5	16	0.09	1
P3	11.76	12.65	0.07	1
P4	26.13	23.6	0.10	1
P5	10.38	11.33	0.09	1
P6	71.63	70.1	0.02	1
P7	21.63	20.51	0.05	1
P8	29.25	29.62	0.01	1

28 of 40

WWDE Data�

Project(s)	Actual (PD’s)	Estimated (PD’s)	MMRE	Within 10%?
P1	13	12.4	0.05	1
P2	17.5	14.2	0.19	0
P3	11.76	11.05	0.06	1
P4	26.13	20.63	0.21	0
P5	10.38	10	0.04	1
P6	71.63	61.26	0.14	0
P7	21.63	18	0.17	0
P8	29.25	26	0.11	0

29 of 40

RESULTS�ANALYSIS

30 of 40

Overall Techniques Analysis�

31 of 40

LIMITATIONS

32 of 40

Limitations

Lesser Number of Projects Available

Limited Managerial Expertise of Experts

Geographical Limitations

33 of 40

CONCLUSIONS

34 of 40

Conclusion

Individual Estimation Techniques

TPE(P) Performed Better

Group-Based Estimation Techniques

Wideband Delphi Estimation Performed Better

35 of 40

FUTURE�DIRECTIONS

36 of 40

Future Plans

Increase the Number of Data Projects

Increase the Number of Experts

More Familiar Experts Could be Selected

Expand the Scope Geographically

37 of 40

REFERENCES

38 of 40

References

Shepperd M., Schofield C. (1997) “Estimating Software Project Using Analogies” IEEE Transactions on Software Engineering, vol. 23, No. 12, November 1997.

Jurison, J. (1999) Software Project Management: The Manager's View. Communications of the Association for Information Systems, 2, pp-pp https://doi.org/10.17705/1CAIS.00217

Moser S., Sellers B. H., Vojislav B. M. “Cost estimation based on business models” The Journal of Systems and Software 49 (1999) pp. 33-42.

Jørgensen M., Sjøberg D. I. K. (2001) “Impact of effort estimate on software project work”, Information and Software Technology 43 (2001) pp 939-948

Moløkken-Østvold K. and Jørgensen M. (2003) “A Review of Surveys on Software Effort Estimation”, In IEEE International Symposium on Empirical Software Engineering (ISESE 2003), September 30 - October 1, 2003, Rome, Italy

Jørgensen M. (2002) “A review of studies on expert estimation of software development effort”, The Journal of Systems and Software (2004) pp. 37-60.

Boehm B. W., Rifkin S. and Jørgensen M. (2009) “Software Development Effort Estimation: Formal Models or Expert Judgment?” IEEE Software, Vol. 26, pp.14-19.

Stochel M. G. (2011) “Reliability and Accuracy of Estimation Process - Wideband Delphi vs Wisdom of Crowd”, 35th IEEE Annual Computer Software and Applications Conference.

39 of 40

References

Basu A. and Coelho E. (2012) “Effort estimation in Agile Software Development using Story Points”, International Journal of Applied Information Systems (IJAIS) - ISSN 2249-0868.

Lee W. T., Lee J., Hsu K. H., and Kuo J. Y. (2012) “Applying Software Effort Estimation Model based on Work Breakdown Structure”, Sixth International Conference on Genetic and Evolutionary Computing.

Mahnic V., Hovelja T. (2012) “On using Planning Poker for estimating user stories”, The Journal of Systems and Software 85 (2012) pp. 2086-2094.

Gandomani T. J., Wei K. T. and Binhamid A. K. (2014) “A Case Study Research on Software Cost Estimation Using Experts' Estimates, Wideband Delphi, and Planning Poker Technique”, International Journal of Software Engineering and its Applications, Vol. 8, No.11, pp.173-182.

Lenarduzzi V., Morasca S., and Taibi D. (2014) “Estimating Software Development Effort based on Phases”, 40th Euro micro Conference on Software Engineering and Advanced Applications 978-1-4799-5795-8/14, 2014 IEEE.

Arnuphaptrairong T. (2021) “Enhancing Delphi Method with Algorithmic Estimates for Software Effort Estimation: An Experimental Study” International Journal of Software Engineering & Applications (IJSEA), vol 12, No.4 July 2021.

40 of 40

THANK�YOU