Tools for
Taming
Tangled Code
Seth Tisue
Scala team
Lightbend, Inc.
@SethTisue (Twitter)
@SethTisue (GitHub)
Chicago Area Scala Enthusiasts
March 29, 2018
me
Scala since 2008
Lightbend since 2015
San Francisco since 2016
who am I to talk?
two experiences:
you
feedback welcome
clean code
of course clean code is well-formatted.
clean code is also well-structured.
internal dependency structure
“internal” — your code.
“dependency” — compile-time dependency
dependencies
structure: goals
modularity
readability, comprehensibility, maintainability
fast incremental compiles
dependencies matter
NetLogo case study: goals achieved
2000: monolithic codebase, dependencies everywhere
2002: separate authoring environment from runtime engine� —> deliver lightweight, embeddable runtime
2004: separate GUI code from headless code� —> support batch operation from command line
2008: open-source independently buildable key components� (whole codebase became open-source later)
2013: separate compiler from runtime engine� —> run compiler in browser on Scala.js
NetLogo case study
after:
before:
tooling can help
two basic approaches:
bytecode analysis
👍 language-independent�👍 mature
source code analysis
👍 high fidelity
👎 bleeding edge
Classycle GUI
Classycle CLI
missing: sbt plugin
ProGuard
...is primarily a JAR shrinker
but in order to shrink JARs…
...it does bytecode-level dependency analysis
if your bytecode has dangling references, it will tell you.
done: bytecode based tools
next: source code based tools
Acyclic
Scala compiler plugin
import acyclic.pkg
import acyclic.file
Sculpt
developed at Lightbend for a client
credits: Seth Tisue, Adriaan Moors, Mark Harrah, Stefan Zeiger
open-sourced January 2018
informational — no enforcement
also a compiler plugin, like Acyclic
Sculpt: core
code from Zinc listens to dependency information compiler emits
the exact same information guides sbt/Zinc incremental compilation
https://github.com/lightbend/scala-sculpt/blob/master/� src/main/scala/com/lightbend/� tools/sculpt/plugin/ExtractDependencies.scala
Sculpt: core
https://github.com/lightbend/scala-sculpt/.../ExtractDependencies.scala
...
...
Sculpt: user-level features
Sculpt: sample output
(show samples in README.md)
Sculpt: missing features
Sculpt: maintenance status
the Scala team will help keep the ExtractDependencies component correct and up-to-date, if users step forward
the rest of Sculpt is… there.�I can review modest fixes and improvements.
Sculpt: status
you might:
(aside on compiler plugins)
my other talk: “Compiler Plugins 101”
video coming soon
IntelliJ Dependency Structure Matrix
Extractor
• https://github.com/CANVE/extractor� looks very Sculpt-like? no work in two years
keep it once you got it
questions for audience
questions for me?
related to talk?
and if we run out of those,
Ideas & TO DOs
Andrew Hill told me Visual Studio has a “dependency map” tool: https://docs.microsoft.com/en-us/visualstudio/modeling/map-dependencies-across-your-solutions�This is very relevant, I could show a screen shot and say man, I wish we had this for Scala.
He also pointed out IntelliJ Ultimate Edition has this: https://www.jetbrains.com/help/idea/working-with-diagrams.html, also very relevant and worth showing at least briefly.
Somebody in Chicago — was it Steven Hicks? – wanted to hear more about remediation, in addition to the material on diagnosis. The talk is called “tools for taming tangled code”, but the material is all about diagnosis — we don’t actually tame anything, we just get insight into its tangledness. The talk is already too long so I hesitate to add a lot of material on this, but maybe at least one slide on techniques I typically used to cut dependencies. By far the most common technique being separating API from implementation. But also using events instead of direct method calls. Instead of A calling into B, A shouts something into the air, and B listens for it.
Ideas & TO DOs
some feedback I’ve received on this talk that I should consider incorporating if I give it again:
Ideas & TODOs
Overall this talk is a bit problematic because it isn’t a super happy story. This isn’t a talk about how tons of amazing perfect tools exist, it's how about there are some imperfect things out there that may be useful if you need them enough. Am I framing this right in the talk abstract and in the opening minutes of the talk…? Also, is the talk too long? If the goal is to call attention to some tools that people might not know about it, maybe it’s not necessary to go into so much detail about them, maybe I could condense the whole thing into 15/20 minutes, still help the people I want to help, and not use up so much of other people’s time. (One person told me I went on too much about NetLogo, perhaps much of that material could be cut.)
On the “keep it once you got it” slide, it’s maybe worth expanding on the “separate subprojects” bullet and emphasizing that this is my most recommended solution for most people. Once you’ve used Sculpt et al to carve out a subproject, afterwards the compiler will keep you honest, and subprojects are a lot easier to deal with than separate repos. If you don’t know how to make subprojects in your build tool, learn! It’s a hugely valuable skill to have. In my early sbt days I shouldn’t have put off learning it for so long.
Do I have examples showing why bytecode analysis is insufficient? Constant inlining, type-level dependencies that disappear in bytecode? I should include them, it should be in the slide deck, even if I don’t dwell on it when speaking. And how insufficient is it, anyway? Maybe it’s actually fine for most people. I guess that’s somewhat covered already by how much time I spend on Classycle in this talk.