Testing, Static Analysis, and Continuous Integration (Cont’d)

Fall 2024

1

Post-Helene Welcome Back

Welcome back! I recognize that the past month has been challenging for everyone, and that each of you have been impacted differently, depending a variety of factors and circumstances. My goal is to help you navigate the remaining course content in a way that is manageable, but that helps you learn some of the core software development competencies that will help you going forward. Before talking about content, I wanted to discuss three items of business:

1. Syllabus & schedule updates
2. Course format and participation
3. What’s left?

Syllabus & Schedule Updates

• The midterm has been cancelled. There will still be a cumulative final exam.
• Select topics have been consolidated, and some of the labs have been shortened.
• Projects 1 & 2 has been simplified / reduced in scope.
• The schedule has been reformatted to be more conducive to remote learning (walkthrough)

​

Syllabus & Schedule Walkthrough

Course Format & Participation

• I will still be taking attendance and doing synchronous lectures during regularly scheduled class time over zoom
• I will record and post all lecture recordings to the course website.
• If you are not able to attend synchronous lectures, you can still earn full credit for “attendance” completing a 1-page reflection that you will submit to Sarah weekly by email (svanwart@unca.edu). Your reflection should answer at least three of the discussion questions listed under the current topic (posted on the schedule).

What’s Left

In my view, you’ve already done a lot of the tricky stuff (Docker, bash, and git / GitHub). Good job! What’s left now is the fun stuff (IMO), including:

• 5 short-ish labs where you will be programming
• 2 slimmed down projects where you will practice your collaborative coding skills
• 1 exam
• 8 more lectures

Office Hours & Scheduling Meetings

If you are concerned that you will not be able to engage with the course assignments as planned, please talk to me as soon as possible so that I can make reasonable accommodations.

For any other questions on labs, projects, or course content, please attend my office hours, which are Mondays, Tuesdays, and Fridays from 3:30pm - 4:30pm on zoom or by appointment.

Questions?

Now onto the course content…

Introduction to the Idea of “Shifting Left”

Prior to Helene, we kind of talked about testing (but I forgot to post the readings so the scope of our discussion was limited).

​

Today – one month later (!!) – we are boarding our testing, static analysis, and continuous integration discussion based on the following three Google Chapters:

1. Chapter 11. Testing Overview
2. Chapter 20. Static Analysis (skim)
3. Chapter 23. Continuous Integration (skim)

Outline

1. Testing
2. Static Analysis
3. Continuous Integration
4. Intro to Lab 6

Outline

• Testing
• Static Analysis
• Continuous Integration
• Intro to Lab 6

Testing: Discussion Questions

Some testing questions you should be able to answer by the end of this unit:

1. What are some of the benefits of automated testing?
2. What are some of the challenges or limits of automated testing?
3. What are the different test "sizes"? Why are these distinctions important?
4. What is meant by "nondeterminism" in testing?
5. What are some important qualities of a testing suite?
6. What are some considerations that go into writing testable code?

​

1. What are some of the benefits of automated testing?

Your answers from Tuesday, September 24…

• Validate that your code works the way you think it should (automate the boring parts)
• More consistent – takes out the human error. Saves $.
• More efficient – can help push out changes faster.
• Keeps colleagues from breaking your code unexpectedly.
• You can be confident in your feature, but how do you know that it didn’t break something else?
• Faster iteration / deployment.
• Helps you write better code

2. What are the challenges / limits of automated testing?

• The tests may not be sufficient – errors can still happen.
• How do you not overdo it, and also maintain all the tests your writing?
• Not everything can be automated
• “You don’t know what you don’t know” – devising tests for things you aren’t aware of can be very difficult.
• Efficiency of your test suite?
• First pass tests should be really fast.

3. What are the different test "sizes"?

• Small tests – 80%
• Run on a single thread (no concurrency, no network calls)
• Small set of functionality
• Fast
• No sleep, wait, no I/O
• Medium –15%
• Multithreaded.
• localhost:// calls OK
• Some file system stuff
• Large – 5%
• Flakey tests: Fails randomly (non-deterministic – network calls).
• Don’t run them on every change. Run before a build or a release.

Outline

• Testing
• Static Analysis
• Continuous Integration
• Intro to Lab 6

Static Analysis: Discussion Questions

Some static analysis questions you should be able to answer by the end of this unit:

• What is the difference between an interpreted and a compiled language?
• What languages are interpreted? What languages are compiled?
• What do we mean by “static”?
• What are some examples of static analysis tools?
• What are some of the benefits of doing static analysis?
• What are some of the challenges / limitations of static analysis?

3. What do we mean by "Static"?

“Static analysis refers to the process of analyzing source code to find potential issues such as bugs, antipatterns, and other issues that can be diagnosed without executing the program.”

4. What are some examples of static analysis tools?

​

• Autoformatters – Focus on formatting code automatically, enforcing style consistency (e.g., Black for python, prettier for JavaScript)
• Linters – Analyze code for possible errors, style violations, or inefficiencies, but typically leaves the code unchanged (e.g., Flake8 for python, eslint for JavaScript)
• Security / Dependency tools – scripts that identify deprecated dependencies or security vulnerabilities (usually written in-house).

5. What are some of the benefits of static analysis?

Your answers here…

6. What are some of the drawbacks / tradeoffs os static analysis?

• There are always exceptions to the linters – a pain to add the skip comment.
• False positives (flagging legit code as problematic)
• They can be a pain to configure.

Outline

• Testing
• Static Analysis
• Continuous Integration
• Intro to Lab 6

Continuous Integration (CI): Discussion Questions

Some continuous integration questions you should be able to answer by the end of this unit:

• What is continuous integration?
• What are some of the key benefits and headaches (i.e. tradeoffs) of continuous integration?
• Can you still use continuous integration if you’re working on a really big feature that’s not ready for prime time?
• What happens in the "presubmit" phase?
• What is release candidate testing? How is it similar / different from the "presubmit" phase?

1. What is Continuous Integration?

“Continuous Integration, or CI, is generally defined as “a software development practice where members of a team integrate their work frequently [...] Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.”1

​

Simply put, the fundamental goal of CI is to automatically catch problematic changes as early as possible.”

2. What is “Shifting Left”

​

The cost of a bug increases exponentially the later it is caught (has it already been deployed, or are we still in early development). Therefore, the more tools you have in place to catch bugs earlier in the development process, the better!

5, 6. What happens in these phases?

What kinds of testing / validation happens in…

1. Presubmit
2. Post-submit?
3. Release candidate?

3. What are some of the benefits and headaches (i.e. tradeoffs) of continuous integration?

main

• sarah-chat-feature (forever to make)

​

Options:

• Feature flag
• ​

Outline

• Testing
• Static Analysis
• Continuous Integration
• Intro to Lab 6

Lab 6: Testing & Static Analysis

https://csci338.github.io/fall2024/assignments/lab06