Workflow optimization is an all-time emerging field in the business. Even so, all companies have problems with efficient workflow. The document presents ongoing investigation into the central goal: “Efficient IT work”. I created this document based on my corporate and startup experience. I’m an IT Leader and Software Expert. However this document is not about software. It’s about work methodology.

In a corporate world I was exposed to the big organization schemas. In startups, among many things, I was responsible for product delivery, system architecture and team building.

Creating an efficient IT structure is very challenging and super important for the business health, work quality and performance. It’s exceptionally difficult in the startup world when you need to be responsive to:

goals and priorities changes
business reporting (especially to people who don’t understand IT work)
management hunger
delivery

Successful execution

Success is an effect of: organization, risk management, focus, open mindset and proactivity. Having the idea is just a small part of the process.

2. Work Principles

This is a learning in the business life that first of all you need to have commitment, dedication and passion for what you are doing.

-- Lakshmi Mittal

The key to effective management of agile teams lies in the mastery of soft skills.

The Team Matters

The main principle is that the team matters. You should respect the team and the work the team is doing.

Be Open

It is important we all know what’s going on behind each other’s screen to some extent. Be open, share with your teammates your achievements and successes as well as your problems and issues. Don’t hesitate to tell when things frustrate you, when something really motivates you.

We are working in a team, hence we have to behave as a team. Always inform your colleagues about your issues, what’s your day plan, when they can count on you. Always inform about non-default schedules.

It’s normal that you have private things to do. But don’t hide it. Others may expect something from you. Let them know that you won’t be available (if it’s something longer).

Use a project management tool (Pivotal tracker, Trello…) to track your work and share the updates and goals with others.

Be Proactive

Always share ideas you think are good! Focus on what you can do, rather than what you can’t. Be proactive ⇒ be positively active & be reasonable = able to make reasoning.

Plan — if you have impediments, issues or something which puts the delivery into a risk - plan it. Proactive people plan ahead. Think about the impact. Discuss it. Best if you can propose some preliminary solutions.

Be honest

This is very important. We can’t plan or communicate well if you are not honest. If you have a problem -- let us know. Otherwise we will fail to make a plan and work as a team. Remember the team performance depends on you.

If you have a personal issue — talk about it with someone you like in the team. It’s better to solve things together.

Be Encouraging

Encouragement is the principal for building an efficient and sustainable environment.

Be Productive

Increase your productivity through better scheduling.

Block off 3 entire days every week for no meetings.
Having a full day free from meetings allows both managers and non-managers to get into their flow zone and tackle the big projects that often get interrupted other days of the week.
Block off the right time for big projects.
A lot of procrastination is a result of poor scheduling: trying to tackle energy-consuming big projects at times when you’re most in need of a break, or not quite ready to think big.
Use the Pomodoro Technique.
Primary, the objective is to focus on solving the task and then recharge.
It advocates breaking big projects or tasks into smaller, more achievable tasks distributed throughout the course of your day and — notably — separated by frequent breaks. Each task should take no longer than 45 minutes and breaks should be approximately 5 – 15 minutes (depending on how long you need to recharge before the next task). The Pomodoro Technique is a time management philosophy that optimizes for creative recharging, so you’ll have less mental fatigue and can move faster through projects.
If a task is not scoped correctly then start with specifying the task.
Schedule 30 minutes to an hour on your calendar at the beginning or end of each day and use this time to organize your day and respond to teammates in your Inbox (and email).
Try to carry out the plan. The most important part is to deliver. Whether you have a great idea or not - all of us want to successfully deliver a new feature set and deliver new releases.

Be Agile

Agile software development is an umbrella term for several software development methods (including Extreme Programming and Scrum). More on this below.

3. Agile

Intelligence is the ability to adapt to change.

– Stephen Hawking

In a nutshell Agile is a time-focused, iterative philosophy that allows to build a product step-by-step (incrementally), delivering it by smaller pieces. One of its main benefits is the ability to adapt and change at any step (depending on feedback, market conditions, corporate obstacles, etc.) and to supply only relevant products to the market.

To its advocates, Agile is a genuinely better way to run a company and an economy—better for those doing the work, better for those for whom the work is done, better for the organization itself. Instead of management extracting value from the firm, Agile generates value for customers and for society as a whole.

To understand Agile we need to look at the company through customer satisfaction and project risk. The picture below shows how it works (compared to the traditional, waterfall approach), and the following sections will explain the mechanics of it.

Structure vs change

Workflow is about creating structure through methodology, process and patterns. Agile is about adapting to change and evolution as you go.

Agile is Lean

Agile methods were influenced by the ideas of lean manufacturing. The main lean principles are:

Build faster → Measure faster → Learn Faster

The Lean startup.

Identifying the most critical parts of your project and going step by step with strong validation between steps is the foundation of Lean methodology. Introduced by Eric Ries, Lean projects focus on interaction with customers and validation (worth watching: How Today's Entrepreneurs Use Continuous Innovation).

How to validate your next idea?

High Level Plan: define User Experience vision - step by step description of the usage and features.
Make a step back and identify any assumption you’ve made, that if wrong would result in a large amount of wasted time and effort.
Start with MVP. Make it really minimal either with semi-manual processes (concierge MVP) or smoke screens MVP (marketing, without functionality). This way you will measure actual user behaviour (and check if they will buy)!
Present to a small segment of your target customers. Then measure their behaviour. Use advertisements. Measure amount of clicks / people watching the video.
Pivot and Preserve. If you don’t reach the goal then you need to change something.
The sign of a successful pivot is that these engine tuning activities are more productive after the pivot then before [E. Ries].

MVP

MVP does not only express the feature set, but also the approach (we should accept that product is not working perfectly, and we should drive the project to reach good enough quality).

Be Agile ⇒ Be Lean

Take Agile as the extension of the Lean.

A typical lean company follows a learn – measure – build cycle, and conducts many tests, frequently connects with customers, understands their value and focuses its key processes to continuously improve it. A never ending cycle leads the teams to sustainability, smart development and success. Let’s put them together:

Agile Philosophy	Lean Philosophy
Execute tasks faster, adapt to changes easier	Smart development, when you improve virtually everything you do by eliminating anything that doesn’t bring value to the customer
Makes the developing process flexible	Makes the developing process sustainable
Was initially designed for Software Development, then expanded to Marketing, and is currently applied in other areas	Started from traditional manufacturing and expanded to all existing industries
Action loop: product backlog – sprint backlog – iteration (sprints) – potentially shippable result	Action loop: build-measure-learn
Method for demonstrating progress — definition of ‘done’	Method for demonstrating progress — validated learning
Methodologies: Scrum, XP, FDD, DSDM, Crystal Methods etc.	Methodologies: Kanban, Kaizen etc.
Toolkit: sprints, boards, Scrum Master, acceptance tests, user story mapping etc.	Toolkit: hypotheses, split (A/B) tests, customer interviews, funnel and cohort analysis, Customer Success Manager etc.

Be Agile ⇒ Empower the Team

The best architectures, requirements, and designs emerge from self-organizing teams.

-- Agile Manifesto

One of the key important aspects of a successful Agile setup is having a self-organizing team.

Self-organizing teams take responsibility and manage their own tasks and don’t rely on a manager to tell them what to do. Team members:

pull work for themselves and don't wait for their leader to assign work. This ensures a greater sense of ownership and commitment.
manage their work (allocation, reallocation, estimation, reestimation, delivery, and rework) as a group.
still require mentoring and coaching, but they don't require "command and control."
communicate more with each other, and their commitments are more often to project teams than to the ScrumMaster.
understand requirements and aren't afraid to ask questions to get their doubts clarified.
continuously enhance their own skills and recommend innovative ideas and improvements.

Development Teams are structured and empowered by the organization to organize and manage their own work.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done

-- Agile Manifesto

4. Use the project management tool efficiently

Projects are lists of tasks. Use a project for a larger goal and a task for an action someone on your team needs to take in order to achieve that goal.

-- asana.com

Example

This is an example. You should be able to use any other project management tool (PM tool) of your choice.

You can use projects to express different goals of your work schedule:

Plan your sprint, milestones and releases
Group tasks per user stories
Additionally you can use tag to generate a tag based view (tag examples: important, difficult, company-x1, …)

Slice and dice your tasks into multiple projects - this way you will create a better overview of the work and features. This idea is nicely presented in Project management, goal setting, roadmaps, pipelines… video.

Projects can go beyond simple task list view. Here are few examples with videos who to organize a project:

Check overview of Project Views tutorial to get ideas about sections, ordering, default views, explore files directory, calendar….
Slice and dice your tasks as mentioned above. Get inspiration from the following presentations:

Tribeca Film Festival plans and runs events (youtube)
Pantheon sets company objectives (youtube)
Autodesk manages editorial calendars (youtube)
General overview of (project management)

Keep your “PM Tool” as it’s your Database of work

It’s uber useful for the team and the company if we can aggregate information (links, decisions, files) in a feature task. This way we will not get lost. Connecting work video tutorial explains it perfectly.

Keep your “PM Tool” organised!

All feature tasks must have a good, informative name - this way it will be easy to search them and find related information.
All tasks you are working on have to be in a sprint [number] section (Sprint project).
All tasks you are planning to work today have to be in your Today section (Personal tasks). All these tasks must have an estimated due date. Keep your updates in comments.
Pro tip: you can add a section in upcoming tasks (eg: postponed, in progress, ...) to better organise your upcoming tasks.
Track a due date. Always!
Become a power user!

Keep the base with your customers.

As a side note, next to the tooling I want to highlight the importance of tracking the customer behaviour and contracts. This means both: the marketing, support and analytics.

Make sure you use the right tools across your company.

CRM

Manage your customers, contacts, and sales with the help of a simple CRM. Customer relationship management is a business strategy that stresses good ongoing relationships with customers. It holds that maintaining these relationships drives growth and profitability. CRM software can help by synchronizing customer communications among business units. Sales, marketing and customer service can all get on the same page. It’s a system of record for touch points throughout the customer lifecycle. Features: customer data management, sales force automation, marketing automation, customer service & support,

Learn From Analytics

You can learn about what makes your customers tick by studying the data produced by your reporting systems. Information from profiles, preference centers, and activity can all be combined to give you a better idea of what they truly want and need from the relationship. Having a strong data foundation is having power because it puts you in position to make smarter business decisions and better anticipate the needs of the customers. This is when your finely targeted and tailored messages are likely to have the most impact.

Analytics is no secret, but still it is one of the more underutilized business tools. Some organizations don’t use anything at all while others fall into the trap of limiting themselves to a single solution.

Examples: Amplitude, Countly, Mixpanel, Google Analytics, Heap

5. Planning

Without a clear plan, you might find yourself running a race without a route.

-- wavelength_blog

Remember - to successfully complete objectives you need to carefully plan and explain them. Don’t make the plan on your own! Plan and estimate correctly!.

Prioritize Maniacally

What you prioritize is what you value as a company. So, the prioritization process is the process through which you determine and express the company’s values. Ensure that you don’t do anything that isn’t the next most important thing that can be done. The clearest sign that smart prioritization isn’t happening is stupid perfectionism; if you see that happening, you know you need an upgrade to “more maniacal.”

Be aware about new, distracted ideas

It’s hard to make meaningful progress when you’re constantly open to changing strategy. On the other hand, rejecting all new ideas is the opposite of fostering the kind of organization that can learn, adapt, and react quickly.

Sprint plan vs roadmap

Roadmap is a high level set of big user-stories. It’s there to define the big objectives and priorities. Roadmap helps the sprint planning process to deliver important features to the user and keep the system in good condition.

Sprint, on the other hand, delivers incremental updates leaned into the roadmap.

Meetings

Principles around meetings:

Each meeting should have an agenda.
Make asynchronous what should be asynchronous and synchronous what should be synchronous
Optimize decision making processes = Minimize time and unproductive state.

Note: each meeting should have a goal and agenda. If the meeting doesn’t bring any value then it means that the workflow requires some reorganization or there is something wrong with management. Make sure that:

As a manager you keep on track with the team progress without calling the sync meetings.
As a developer you keep the sprint plan and communicate all impediments, delays and success without waiting for a meeting.
Call the meeting if it’s required and it will contribute to the success story. Before calling for the meeting to communicate the idea using the established channel (eg new task on ideas / meetings project).

Severity issues

Each project has to plan for handling different severity issues. It’s good to define them upfront:

Sev0 outage
Sev1 critical
Sev2 error
Sev3 warning

The IT teams have to plan together how to communicate and share the experience.

Communicate, communicate, communicate

In case of outage and critical error we have to have a channel where everybody can jump in easily and help. Slack #showstopper is the minimum. You need to have a notification channel with mobile support which will notify right persons, something like Pagerduty.

Postmortem reports

After each problem, create a postmortem report. Share it with your team and make sure everybody learns from it. Try to mitigate similar problems by implementing solutions (tools / processes).

Processes / Phases

Leading by example, Atlassian agile teams form three product divisions: make, sell, and operate.

Each division constitutes 3 phases managed by separate teams:

Division	Phase	Focus
Make	Product Management	Understand the market, targeted customer personas, and good product design principles
	Design	Define the value proposition, product goals, and minimum viable product
	Development	Develop the product using sound, sustainable engineering practices
Sell	Product Management	Understand the product's competitive landscape and market evolutions
	Design	Create messaging that highlights the product's value propositions to each customer segment
	Marketing	Build collateral to support the product launch: web pages, announcement emails, blogs, videos, etc.
Operate	Product Management	Release software to customers with a regular cadence
	Development	Respond to customer issues
	Support & Ops	Relay customer feedback to the “make division” (Dev, PM, Design) as input for future product development

The “Make” Division (development):

Scoping (planning) is done in one sprint ahead of design. Design is done one sprint ahead of development.

Planning / Scoping

The goal of scoping is to define all stakeholder objectives. Plan ahead for planning!. The project leader has to describe what he wants and why. He has to involve business stakeholders to make sure that the objectives are in line with the product goals, and clear and extensive to think about the solutions.

Create grooming sessions during the sprint to carefully scope and design next features.

Story Points

It’s essential to apply good measurements for successful planning and product delivery. In Lean Principles, measurements take a central place. Measurement covers many aspects: from product usage, team efficiency, go-to-market and finally measuring the correctness for planning. Story points are a great way how we can measure the complexity of a task. This works on every level: roadmap planning, sprint planning, release planning. Read here how the story point system works: Story Points vs Hours: Reasons to Estimate with Story Points

Finally we need to have a right scale of story points. The past experience taught me that applying a non linear scale (example: fibonacci sequence) works much better than a linear scale. Why the Fibonacci Sequence Works Well for Estimating.

TL;DR: The Fibonacci sequence prevents estimates from being too close to one another.

Weber’s Law states that the difference we can identify between objects is given by a percentage.
Example: Imagine instead being handed a 20kg weight and a 21kg weight. They are the same one kg difference as the one and two kg weights. But you would have a much harder time identifying the heavier of the two weights.

Humans are bad at estimation but good at comparison. Let’s say, if I ask you to tell me the distance between the door of your room and the window, by just looking at it, I am sure you will be able to take a fairly accurate guess of it in the unit of feet. You may be off by few inches here and there. But if I ask to you tell me the distance between New York and Washington then high level ball park distance in miles are sufficient, the expectation is not to be accurate up to a great degree.

Design

During the design phase we want to define user and system requirements, describe the user stories. This process has to involve IT (architect and developer) to be sure that the design is aligned with the system architecture and define the correct timeline for it.

Design thinking.

Following the lean approach there is a “Design thinking” strategy. It is just a way of approaching problems that focus on a human being. For instance, let’s all agree and validate what the problem even is first. Then it’s stuff like mapping, prototyping, and validating assumptions as a group — because all parts of a product team (business, design, and engineering) need to be involved. Start with a basic sketches approach (napkin style). Evaluate ideas and solutions when they are still on napkins. You can’t focus on the important things (like, is it solving the problem? Do we have a hierarchy of actions to be taken?) when you’re distracted by colors and button styles and typefaces. It is an absolute mistake to jump straight to high-fidelity visual design or “re-skin” a prototype that has already been made, or so help me, make a logo for a product with no defined experience. Christoph Meinel and Larry Leifer, of the HPI-Stanford Design Thinking Program, laid out four principles for the successful implementation of design thinking:

The human rule, which states that all design activity is ultimately social in nature, and any social innovation will bring us back to the 'human-centric point of view'.
The ambiguity rule, in which design thinkers must preserve ambiguity by experimenting at the limits of their knowledge and ability, enabling the freedom to see things differently.
The re-design rule, where all design is re-design; this comes as a result of changing technology and social circumstances but previously solved, unchanged human needs.
The tangibility rule; the concept that making ideas tangible always facilitates communication and allows designers to treat prototypes as 'communication media'.

You are not your user.

Obviously you need to research and test quite a bit. Remote testing and recruitment tools have come a long way in the past four years, so there’s no excuse. The most effective strategy is participatory design, bringing users into our workshops. Not only can you watch them use things, they can give immediate feedback, and ideate with you.

Put yourself into the customers’ shoes.

This is very important for the product owner role. We are creating a product for a customer, not for yourself. So always think about the customer.

Organizationally, we need to understand their business, markets, industry, key strategies/drivers, key challenges, how things get done within the company, and more.

Individually, we need to understand what makes them tick, what they worry about, what their personal goals/ambitions are, how they are measured, how they spend their days, and more.

The better we are at connecting with them–where they are, the more effective we will be in identifying how our products and solutions might help them achieve their goals.

We want to be able to mirror their experience, sharing ideas and engaging them in relevant conversations about them and their organizations. The more we understand and can be empathetic, the more effective we will be in engaging them in meaningful ways.

Caution

Don’t overdesign! Working software over comprehensive documentation!

Collaboration

Collaborating well is critical in this space where it’s highly unlikely any one person in the team understands all areas perfectly. Often teams are remote (decentralized!) and so you have to come up with new strategies to create that “everyone in the same room” dynamic. Old school paper works well with a camera!

Development

Development starts with the analysis - collecting the solution ideas and finding possible obstacles which haven’t been defined during the design phase. For non-trivial tasks, a developer will present his solution idea to the rest of the development team for review before coding the solution. If it’s required, a developer works closely with the business to implement the solution and refine the spec document.

Be clear who owns what part of the work. This way we encourage accountability and empower every contributor to play their part in achieving the broader goals.

Working software over comprehensive documentation

Here is a typical development process: from research to synthesis and implementation. The further up with this chain you go the worst Agile methodology works by itself. Remember: the main goal of agile is to provide working software in a proactive environment, not define long term strategy.

Anticipate problems

For more complex tasks, it’s good to do incremental solution idea review. The developer comes with his solution idea and the review is a brainstorm of new ideas and roadblocks. The obstacles you will find can often be gleaned from past failures. Other times, they’re determined by vividly imagining just how certain scenarios will play out

Documentation

The documentation, data and code language is English.

Documentation has many forms: code comments, tests, diagrams, spec documents… In the lean environment the principal artefact of the work is a product. Although much documentation can be replaced with highly readable code and tests, in a world of evolutionary architecture it's important to record certain design decisions for the benefit of future team members and for external oversight. Therefore it’s a good habit to organize the list of objectives, requirements and user stories from the scoping / planning sessions. Another form of the documentation is Lightweight Architecture Decision Records. At some point it’s enough to store information in the project management tool → Keep your project management tool as it’s your Database of work.

To model the feature design use:

Flowchart for modelling simple processes or architecture. You should know Flowchart objects usage (eg from this tutorial).
SysML (UML fork) for the complex diagrams.

Diagrams and specifications have to be stored in a wiki tool (or shared storage). Diagrams should be created using tools like draw.io

6. Shape Up

Shape Up is for product development teams who struggle to shape, build, and ship. Written by the innovators behind Basecamp.

“For one, we’re not into waterfall or agile or scrum. For two, we don’t line walls with Post-it notes. For three, we don’t do daily stand ups, design sprints, development sprints, or anything remotely tied to a metaphor that includes being tired and worn out at the end. No backlogs, no Kanban, no velocity tracking, none of that. We have an entirely different approach. One developed in isolation over nearly 15 years of constant trial and error, taking note, iterating, honing in, and polishing up. We’ve shaped our own way.”

Jason Fried, Basecamp

Below I’m copying a brilliant summary of Shape Up by Sachin Rekhi.

Shape the work

Before even considering a project for building in an upcoming cycle, the Basecamp team always shapes the work. The idea is to make an abstract project idea more concrete by defining key elements of the solution before betting on it. A small group of senior members work on shaping projects in parallel to all the execution teams that are focused on delivering projects that have already been shaped and approved.

The key with shaping projects is it isn't equivalent to putting together a product spec for the project. It's instead at a higher level of abstraction: concrete enough that the teams know what to do, yet abstract enough that they have room to work out the interesting details themselves. As part of shaping a project, you might put together a fat marker sketch, which is a low-fidelity UI sketch of the solution that conveys a high-level design without actually defining the details. You also define an appetite, which is how much time you want to spend on this project based on how much you believe this project is worth in terms of customer and business impact. This is opposed to simply coming up with an estimate of how long the project will take. Each of these elements ultimately gets put together in a short pitch, a document that presents the problem, the appetite, and the solution of the now-shaped project idea for consideration in the roadmap.

The reason this approach is compelling is it allows you to better understand the consequences of taking on a potential project during roadmap planning. Too often in roadmap planning we approve high-level projects like "Redesign the homepage", "Add a calendar view", etc without fully understanding them. We leave it to the teams to figure out the details and deliver a solution. The challenge is after approving the project, we might not like the ultimate outcomes. The finalized design might do far more damage to the overall product experience than good. Or the project might overrun it's initial estimate, derailing all our other plans. By shaping the work upfront and understanding what the potential solution might be before deciding to take on the project, we significantly reduce the uncertainties associated with accepting the project.

Betting table

In a roadmap planning meeting they bring together a few senior leaders at the company (CEO, CTO, head of strategy, and senior eng leader) to decide the projects for the upcoming 6-week cycle based on the pitches that were prepared to be reviewed. Everyone has already had a chance to deep-dive into each shaped project by pre-reading the pitches. This allows the discussion to be very focused on debating each pitch and deciding on it's inclusion relative to the priority of the other pitches. During a betting table meeting, they may debate for each pitch how important the problem is worth solving, debate whether the appetite is the right one for this problem, and even debate whether the proposed high-level design is an attractive solution.

This process ends up allowing the senior team to be far more involved in ensuring the right projects are green-lit based on a holistic view of the problem being solved, the resources required to solve it, and the elegance of the solution as opposed to taking a far riskier bet on a raw idea and simply hoping the team comes back with a properly scoped great solution. It's also far more cost effective than making these calls in late-stage product reviews where it's so expensive to make any meaningful changes to the product when it's already begun implementation.

The process also ends up being far lighter-weight than a traditional spec review process that often precedes implementation of a project. While this traditional process does allow the senior team to review critical elements of a proposed solution before implementation, the fact that it's heavier-weight means that it's traditionally only used for large endeavors and usually skipped for small feature implementations, which usually don't enjoy senior oversight and consideration, which can result in feature implementations that aren't cohesive across the entire experience nor appropriately scoped relative to the level of impact.

Hill charts

The Basecamp team has also reinvented ongoing project status by creating an alternative mechanism for sharing how far along each project is. Traditional project management status struggles with a few issues. Some teams simply look at the number of tickets for a given project and chart the completion of those tickets to provide status. Of course not every ticket is created equal so this gives you a poor sense of project progress. Even if you added estimates to each ticket, as projects progress, more tickets are created as new implementation details are discovered or new bottlenecks are understood. This results in an increase in the number of tickets, which means any initial estimate you were making based on the tickets is still wrong. While you can spend significant time upfront coming up with all implementation tasks and scoping a project, this is often wasted effort when its more realistic to discover detailed tasks as the project progresses.

Basecamp throws out these traditional approaches to ticket-based project estimation and status and replaces it with a hill chart, which is a diagram showing the status of work on a spectrum from unknown to known to done. Early in a project, when you are going uphill, you are still discovering all the edges of the work, creating lots of new tasks as you progress. Once you've understood the full solution, you move from the unknown part of the project to the known part of the project where you're simply executing against the tasks that are well-defined and understood. What Basecamp does - it creates a hill chart for each project and plots each project scope, or component, along the hill chart, giving you a high-level view on where the team believes it is on each of the project components. Leaders can see the current status, but more importantly, track movement across the hill charts. Project scopes that remain stuck are areas for leaders to drill in to see if they can help resolve the bottleneck.

This far simpler project status approach requires far less effort to keep up-to-date but still provides leaders with the transparency they need in order to understand where projects stand and be able to offer to help when needed.

7. RACI

Effective team organization is crucial for the success of any large project. For example, if a team is responsible for planning a new project, who is responsible for determining the core objectives? Who will make operational decisions?

RACI is an acronym for Responsible, Accountable, Consulted, and Informed.

R: Responsible

Who is responsible for getting the work done?
This is the individual who executes a task, also known as the ‘hands on’ individual.
They will generally report to a manager.

A: Accountable

Who oversees the task?
This is the individual who ensures the work gets done properly.
They are not hands-on, but rather responsible for managing the people who are executing tasks and ensuring that work is completed on time and at the required quality.

The difference between being Responsible and Accountable is that Responsible refers to the person or team completing the work, while Accountable refers to the person who is ultimately accountable for the outcome and must report on and sign off on the deliverable. The same person can hold both roles, but they are distinct responsibilities.

C: Consulted

Who needs to assist the competition of a task with additional information or support?
This individual is not directly responsible for a task
However, they do provide information that assists the person responsible for a task
They are generally consulted for their expertise in a specific area

I: Informed

Who needs to be kept up to date on the progress of a task or deliverable?
This is generally an upper management stakeholder or potentially a client.
They do not have immediate input on the project but may be the project’s owner.

8. Extreme Programming

Starting with some framework from the very beginning is uber important.

There are many good methodologies applying Agile principles. I’m in favour of the eXtreme Programming (XP), which works very well in the startup environment. Another methodology, which works especially well in the enterprise environment is Scrum.

XP focuses heavily on ensuring the quality of delivered software which prescribes engineering solutions towards that end. It solicits feedback immediately - you don’t have to wait for the new sprint to tackle the feedback.

XP Ceremonies

The team engages in Release Planning and Iteration Planning. They work in very short development cycles so that changes requested by the customer (who works on-site with the team) can be incorporated frequently.

Quarterly roadmap review.
Time to dream big and review all ideas we wrote down over the past months.
Make a few events: one for your team and one for the business and development together.
Weekly sprint planning.
Goal: define the sprint tasks you will commit on.
Daily stand-up.
Goal: quickly find what’s blocking. Help and support.
In a distributed environment can be done in a short report form in your chat app. Each morning, when you start a work you communicate in 3 paragraphs:

DONE: what have you done.
DOING: what are you planning to do.
BLOCKING: what is blocking you, what’s not clear. Describe your problems.

Even if something was discussed, it’s better to repeat yourself in daily updates than not doing it at all.

Keep on track with the sprint plan.
Work principles: Keep on the sprint plan. Measure. Communicate!
Weekly sprint review.
Everybody will present their work (shortly, max 3min per person).
Retrospective every 2 weeks.
Discuss process improvements, successes and failures.
The team is obligated to note their comments during the sprints. Organize them in 4 categories:

Start doing
Stop doing
Do more of
Do less of

Make a document retrospective_sprint_id, where we will share all our comments at the beginning of the retrospective. Start each retrospective by reading the previous retrospective document.

XP Release planning

Start out collecting user stories.

After user stories have been written you can use a release planning meeting to create a release plan. The release plan specifies which user stories are going to be implemented for each system release and dates for those releases / milestones. It is important for technical people to make the technical decisions and business people to make the business decisions.

The release plan is then used to create iteration plans for each individual iteration.

The base philosophy of release planning is that a project may be quantified by four variables; scope, resources, time, and quality. You can quantify maximum 3 out of 4 variables.

Sprints / iterations

Each iteration is 1 to 2 weeks long. Begin your iterative development with an iteration planning meeting. Selected user-stories are translated into individual programming tasks to be implemented during the iteration to complete the stories.

Don't schedule your programming tasks in advance. Instead have an iteration planning meeting at the beginning of each iteration to plan out what will be done.

It is also against the rules to look ahead and try to implement anything that it is not scheduled for this iteration.

Testing Rules

All code must have unit tests.
All code must pass all unit tests before it can
be released.
When a bug is found tests are created.
Acceptance tests are run often and the score is published.

System Metaphor

It is a metaphor for a simple design with certain qualities. The most important quality is being able to explain the system design to new people without resorting to dumping huge documents on them. A design should have a structure that helps new people begin contributing quickly. The second quality is a design that makes naming classes and methods consistent.

What you name your objects is very important for understanding the overall design of the system and code reuse as well.

Define success metrics

Recall the Lean Principle:

Build faster → Measure faster → Learn Faster

Data can be an incredibly powerful communication tool, but even the most informed product teams can skew or misunderstand the purpose of a new feature because of a miscommunication.

Data-driven planning

As a Project Leader, during Scoping and Design stay clear about the most important goals that can be solved during the sprint. Drive the user stories using the data (metrics). Establish baselines by gathering metrics and tracking how new features and processes impact them.

However, sometimes the data won’t be able to tell you how to prioritize and scope or why the metrics are shifting (doing opposite than expected). Rely on your best judgment, test often, and build in frequent reviews to see what tactics are causing metrics to shift in the direction you want.

KPI for the Roadmap.

It’s crucial to define KPIs for your roadmap. What brings the value to the customers? Does the system work seamlessly? Measure completeness of your roadmap and KPIs for each sprint planning.

Security

Development teams are under pressure. Quality releases needed to be delivered on time. Coding and compliance standards need to be met. And mistakes are not an option.

Make sure you integrate sufficient measures (practices, methodology, tools and processes) to ensure system security.

Treat Model

Threat modeling is a type of risk analysis used to identify security defects in the design phase of an information system. Threat modeling is most often applied to software applications, but it can be used for operating systems and devices with equal effectiveness. Threat modeling is typically attack-centric; threat modeling most often is used to identify vulnerabilities that can be exploited by an attacker in software applications.

STRIDE technique (list of basic threats):

Spoofing of user identity
Tampering
Repudiation
Information disclosure
Denial of service
Elevation of privilege

Read: 12 methods for threat modeling

Static Code Analysis

Static code analysis is a method of debugging by examining source code before a program is run. The sophistication of the analysis performed by tools varies from those that only consider the behaviour of individual statements and declarations, to those that include the complete source code of a program in their analysis. The uses of the information obtained from the analysis vary from highlighting possible coding errors (e.g., the lint tool) to formal methods that mathematically prove properties about a given program (e.g., its behaviour matches that of its specification).

Limitations:

No understanding of developer intent
Rules that aren’t statically enforceable
Many false negatives (not discovered vulnerabilities).

Example tools:

https://www.checkmarx.com/

Below we will list common techniques..

Data Flow Analysis

Data flow analysis is used to collect run-time (dynamic) information about data in software while it is in a static state (Wögerer, 2005).

Control Flow Graph (CFG)

An abstract graph representation of software by use of nodes that represent basic blocks. A node in a graph represents a block; directed edges are used to represent jumps (paths) from one block to another. If a node only has an exit edge, this is known as an ‘entry’ block, if a node only has an entry edge, this is known as an ‘exit’ block.

Taint Analysis

Taint Analysis attempts to identify variables that have been ‘tainted’ with user controllable input and traces them to possible vulnerable functions also known as a ‘sink’. If the tainted variable gets passed to a sink without first being sanitized it is flagged as a vulnerability.

Lexical Analysis

Lexical Analysis converts source code syntax into ‘tokens’ of information in an attempt to abstract the source code and make it easier to manipulate (Sotirov, 2005).

9. Weekly Business

It’s very important to keep weekly business meetings on the product lever.

A very good template for such meetings is following:

New Business: agenda of new things to do. (15min)
Old Business: executive summary of the ongoing project updates. (15 min).
Round table (30min, max 5min / person): each person has his time share something that matters.

While going through the 3 stages listed above, we need to note items in 3 categories

In Attendance: list of people who attended the meeting.
Parking Lot / Deep Dive: broader scope, to be discussed in a breakout session.
Actions: action items.
Decisions.
Roadblocks.

10. Growing: multiple teams

Over time, once the development team grows above 8 people we should form multiple teams.

An agile team must possess all the required skills (cross functional agile teams), but sometimes it's necessary to call on specialists for specific work.

By definition:

An Agile team is a cross-functional group of people that have everything, and everyone, necessary to produce a working, tested increment of product.

Dedicate these people to the team, and as a rule, do not move them between or across teams as demands ebb and flow.

Cross-functional team and product definition

Being cross-functional doesn’t mean that the team has to be able to do everything (design, frontend, backend, analytics …). We need to finely define the product the team is building. A product is actually a sub-system of a larger systems-integration.

Cross-functional means that a team has to be able to build the product.

Dependencies

Dependencies are unavoidable. They are always there. Don’t try to fight them. No amount of daily standup meetings is going to fix this problem. Instead learn how to:

Break dependencies
Plan dependencies

Keep the following as a rule of thumb:

The more you manage dependencies the less Agile you will be.

Manage Constraints Rather than Dependencies

The solution lies in the application of Kanban to model the flow of value across teams, to make smaller investment decisions at the portfolio level, to limit the amount of work in process, and to redeploy people and teams in ways where everyone all the time, is focusing on the highest value initiatives within the organization. We use agile at the team level to inspect and adapt and to make sure we are always focusing on delivering the most value possible in any given sprint. Using Lean and Kanban and Theory of Constraints gives us that same ability when we are dealing with dependencies at any level of the organization.

Forming a team

Agile teams go through four key phases as they develop:

Keeping agile teams intact takes some organizational discipline, but it pays to protect the team–within reason, of course. When change is introduced (new hire, employee departure, etc.), the team reverts back to the forming stage as it absorbs the change.

Site Reliability Engineering

Communicate this across all organizations (from devs to presidents).

SRE is a prospective way of accomplishing DevOps philosophy. SRE implements DevOps.

DevOps	SRE
Reduce Organizational Silos. By breaking down barriers across teams, we can increase collaboration and throughput.	Share ownership of production with developers. Use the same tooling in order to make sure everyone has the same view and the same approach.
Accept failure as normal	Blameless postmortems. Use a concept of error budget of how much the system is allowed to go out of spec.
Implement gradual change. They are easier to review, it reduces the time to recover making it simple to roll back.	Canary things. Roll things out to a small percentage of the users before moving out to all users.
Leverage tooling and automation.	Eliminate manual work as much as possible. Measure how much toil you have. Automate this year’s job away.
Measure everything.	Measure that amount of toil you have, reliability and health of your systems.

SREs focus on solving classic problems: developers want to push and release new features. Operators aim for stability and maintenance. Speed vs reliability.

For SRE we have 2 categories of responsibilities:

Reactive work is looking at operations requests if somebody needs help.
Proactive is about improving the system so they are more reliable and easier for people to use.

Service Level Objective (SLO) and Service Level Indicators (SLI)

How to come up with a precise numerical target for system availability? We term this target the SLO of our system. It’s an agreement among stakeholders about how reliable a service should be. Cost and technical complexity of making service more reliable gets higher and higher the closer to 100% you try to get. Make sure you have enough room for error and enough room to roll out features reliably. To do this, you need to define:

Availability. Have a clear numeric indicator. What does it mean to be available? Successful response vs consistent response. You do this by defining Service Level Indicators. These are metrics over time such as: request latency, throughput, failures per request over time.
What proper level of availability is in context of SLI. Add all the SLI up or integrate it over a longer period of time (eg: a year) to see if the total amount of errors/downtime that we’ve had more or less the specified SLI.
Plan in case of failure. Put everybody on the same page.

SLO are both upper and lower bounds. This is for two reasons:

If you try to run your service much more reliably than it needs to be, you are slowing down the release of features that you want to get out and make your customers happier.
It’s an expectation that you are sending for your users, that if you suddenly start breaking a lot more often they are used to because you start running exactly at your SLO rather than doing much better than your SLO, than users will be unhappily surprised if they’re trying to build other services on top of yours.

Service Level Agreement (SLA)

It says: here is what I’m going to do if I don’t meet the level of reliability that is expected ~ business agreement associated with SLO.

SLIs drive SLOs which inform SLAs

SLAs describe a set of services and availability promises that a provider is willing to make to a customer, and then the ramifications associated with failing to deliver on those promises. Those ramifications might be things like money back, free credit...

Example:

SLI: 95th percentile latency of homepage requests over past 5min < 300ms.
SLO: homepage SLI will succeed 99.9% over trailing year.
SLA: Service credits if homepage SLI succeeds less than 99.5% over trailing year.

Google Cloud Platform Blog provides nice explanations at the SLOs, SLIs, SLAs, oh my - CRE life lessons.

Risks and error budgets

Accept failure as normal by quantifying failures and risks through an error budget. It enforces gradual change, because non-gradual change could quickly burn all error budget breaking the SLO.

SLO defines error budget. If we approach SLO we need to reduce the risk and not accept risky features in order to balance innovation and stability to an appropriate level.

10. Architecture Decision Record

An Architecture Decision Record (ADR) is a document that captures a decision, including the context of how the decision was made and the consequences of adopting the decision. It helps with: new team members onboarding, ownership handover, alignment and clearing out the context of a decision process (it’s very often that we either forget the main factors or it’s lost in tonnes of discussion / emails / issues).

An ADR should be written whenever a decision of significant impact is made; it is up to each team to align on what defines a significant impact. This relates to:

backfilling decisions
proposing large changes (with that matter, an ADR should be proceeded with an RFC)
Proposing small changes

For more please the ADR guidelines from spotify.

Still not convinced if your Boss will buy it?

Agile ⇒ Lean: Build faster, Measure faster, Learn Faster

source: http://dilbert.com/

Table of Contents

1. Introduction