CSA Training

Who am I/Why am I qualified for this?

• Amalan Iyengar
• EE at Johns Hopkins University Applied Physics Lab
• Communications and Networking group
• 888/4795 Mentor
• CSA since 2017, Robot Inspector since 2018
• Started as a 4795 student by building robots that barely moved
• We (and I) have gotten slightly better at this
• Passionate about getting every team driving and scoring

​

​

Outline

• What is a CSA?
• Customer Service Mindset
• Soft Skills
• You are an Advisor
• Helping teams through difficult choices
• Technical Stuff
• Resources

What is a CSA?

• Control System Advisor - Orange hats
• Helps teams with anything control system related
• Electronics
• Code
• Field connection
• Works closely with inspectors and FTA/As to assist teams
• An advisor
• No authority over teams like an inspector or FTA has
• Puts you in a valuable position to help teams and build trust
• You don’t have to know it all
• Electrical-only or software-only CSAs are still helpful!
• Rely on other CSAs, Inspectors, FTA/As, and teams to help fill the gaps
• The job: Keep students inspired, get teams on the field
• The dream: Every team playing in every match

Customer Service Mindset

• Teams are the paying customers
• We are volunteers there to serve them, students and mentors
• Everything else is secondary
• When you’re with a team, you are with that team - pretend you’re a member
• Bring your GP
• Remember - this may be the most pressure a student has been under
• CSA training has slides on customer service training - read them!

Soft Skills

Soft Skills

• The hardest part of CSAing
• Technical stuff is learned from being an alum, mentor, inspector, CSA, etc
• CSAs work with extremely stressed students and mentors under severe time pressure
• Tough, emotional decisions to be made
• “Are we playing this match”
• “Are we removing mechanisms to get driving working”
• Be a beacon of calm
• Missing a match isn’t the end of the world - make sure they know that
• Students will match your energy
• Have to provide technical guidance while making the students feel better
• Know when to ask for help
• Your job is not to fix issues yourself, it’s to keep students inspired and get robot on field

You are an Advisor

• Control System Advisor: The only volunteer with zero power over students
• Students are naturally scared of authority
• Inspectors can tell them robot is illegal, FTAs run the field
• CSAs should be friendly neighborhood orange hats - holding nothing but desire to help
• Lack of authority over teams gives a unique opportunity to build trust
• Liaison to field staff
• If a connection issue, make sure they know you and the field are working it
• Teams can have a “Team vs. Field” view
• We know it’s not true, but they don’t
• Help them understand it’s not true and make sure they know you’re in their corner
• If you’re an alum - remember how it felt for you
• Can feel to the one electrical student that it’s all their fault the robot isn’t working
• Make teams feel heard and supported

Working with the Team

• Ask the team what they want and make judgment calls
• Ask before touching robot or doing stuff
• Sometimes teams just need you to fix so they can play
• Other times, walk them through how to make the fixes
• Sometimes, teams know exactly what’s wrong and don’t need you!
• Ideal scenario: Use their tools, teach them as you go
• Explain as you read the logs what you’re looking for
• Do the first connection/tug test, transitioning to students doing it
• However...
• If it’s a matter of making the field or not, and the team is OK with it, you can just do it
• Put yourself in their shoes
• New for 2024: Nexus app, monitor that for notifications
• Teams know their robot best but can miss obvious things

What NOT to do

• Do stuff on robots without asking
• Judge teams
• Everyone’s got different circumstances, it’s OK
• Belittle students or ask “why would you do that?” type questions
• You will see some mind-bogglingly silly stuff
• Save the ranting for later, just help the team without judgment
• Come off as superior
• Assume you know more than the team
• Help teams who don’t want to be helped
• Even if sometimes they really should want the help - it’s OK

Technical

Resources

• CSA Training - will be sent to you when you volunteer
• WPILib Intro to Wiring
• WPILib Basics
• Robot Reliability

Debugging Process

• Starting points:
• Ask drive team “what did you observe/how did it feel”
• Check Driver Station logs
• Can use built-in DSLog viewer, or AdvantageScope (comes loaded with WPILib now)
• Ask if they’ve seen it before/how other matches were
• If software issue - look at console output, go through stack trace
• Make sure student programmer is with you
• If they don’t have their programmer or laptop... get ready to write some on the fly
• If electrical issue: most problems solved with tug test and correctly stripping/crimping wires
• Check blink codes on all devices
• If connection related - talk to the FTA/As, ask to see their logs if issue isn’t obvious

Debugging Process Cont

• If it isn’t obvious:
• Isolate the issue - make sure the individual component works
• First electrically - power, then communication (CAN/PWM), then software
• Build out to the system level
• Ask for help!
• Prepare a swap in parallel if needed
• E.g. if potential RoboRIO issue, flash a spare while debugging
• Tug test and wiggle all the wires

Reading DSLogs

• WPILib Docs cover this better than me
• Use DSLogReader2 (in the CSA USB tool) or AdvantageScope to make life easier
• Checking logs is almost always one of the first steps
• Shows voltage, current draw, errors, brownouts, CPU util, radio status, etc

REV Hardware Client + Phoenix Tuner

• Your best friends for solving motor-related issues
• Show error codes, let you run individual motors, show what is connected to bus, update firmware
• Can help you isolate motor control issues
• Get familiar!

Common Issues

Radio Drop

• Symptom: Robot dead for about 45 seconds
• DS will show no link to Radio or RIO
• Field will show no Radio
• Fix/Mitigations:
• Make sure team is using Power over Ethernet
• If using VRM, do redundant barrel jack and Ethernet
• Tug test/make sure no exposed wire in radio power line
• Other radio things: Port 2 bad, teams should use network switch
• Jittering comms to Rio, but radio good - check Ethernet wiring between radio and RIO

​

Rio Drop

• Symptom: Robot out for ~20 seconds
• DSlog may show no comms to Rio, but good comms to Radio
• Also may show a code error
• Fix
• If no code error and DS loses connection to Rio, check power and ethernet wiring
• If code error, open console and check the stack trace
• Common errors: Null Pointer, Divide by Zero
• Stack trace will lead you to the issue
• New(er) issue: Out of Memory Error on Rio 1
• Remove unused auto paths
• Disable webserver, replace lvrt
• Note - replacing lvrt makes reflashing rio harder

Brownout

• Symptom: Robot and RSL jittering on field
• DSLog will show battery voltage dipping and brownouts
• Show as 12V faults
• Mitigations
• Check power path
• Tug test breaker, battery, PDH power connections, all should be rigid
• Check battery used - should be purchased in last two years ideally
• Test with battery beak - good batteries should be 120%+
• Check current draw
• Look at each channel - especially drive motors
• Is anything drawing insane amounts of current?
• Add current limits
• 60-80A max for NEO/Kraken
• 20-25A max for NEO550
• New motors without limits can cause brownouts with perfect batteries and wiring

CAN Issues: Wiring

• Symptoms:
• Some motors not working
• Driver station reporting CAN timeouts
• Motors jittering
• Isolation & debugging
• Check which motors are giving CAN errors in DS
• Blink codes
• Open Phoenix Tuner/REV HW client to see which devices are visible/showing errors
• Rapidly changing status light on REV often implies yellow/green flip
• Check terminating resistor on PDH
• Measuring resistance across green/yellow should be 120 (+/- a few) with one side of bus connected, 60 with both
• If all else fails: go one device at a time

CAN Issues: Utilization

• Symptoms:
• Robot jittering
• Laggy control
• REV: position/velocity data drop to 0 briefly
• High and varying CAN utilization on DSLogs
• CAN Tx or Rx errors on RoboRIO or sticky faults on CAN devices
• Fix:
• Reduce utilization (CD post to come)
• Fewer devices on CAN bus - more on PWM
• For CTRE devices: CANivore
• More and more common with more motors on the bus - unadjusted CAN traffic gets to be an issue past 14ish motors on RoboRIO bus
• Can util% past 80 or 90 causes issues

What about spooky things?

• Some issues feel intractable
• Some are!
• Field-related issues
• Crazy edge cases nobody’s seen before
• Chances are, someone can help
• If not, start asking “What’s the least this robot can do to be successful”
• If you cannot get a mechanism to work - would they rather go without it or keep working on it
• If nothing else to do - make sure they feel you’re in their corner
• Teams need to feel heard
• You are their liaison to the field staff
• Be an advocate

2023 MDTIM Visualized

Who to ask for help?

• Work with inspectors - they will know who needs help
• Other CSAs, FTA/As - FTA is Field Technical Advisor
• Blue polos, run the field
• FTAAs are FTA Assistants
• Other teams - students and mentors
• I bug 614 mentors all the time to help out
• Can always find me if I’m not CSAing
• CSA Slack
• Very responsive about issues
• ChiefDelphi
• Remember - Spare Parts exists with spares of key control system stuff

What to Bring?

• Try and bring yourself - or if mentoring a team know where they are
• CSA-USB
• USB drive with everything you need
• WPILib installers, software libraries, hardware clients, firmware, etc
• USB A-to-B cable for flashing RoboRIOs, USB C cable for REV devices
• WAGO 221s and CAN lever nuts for quick wiring fixes
• Working on a CSA Kit with all the things needed
• Has plenty of tools and consumables like connectors

Miscellany

• You will mess up, it’s OK
• I advised a team to do something that resulted in them not playing in playoffs match
• Was not the right call in hindsight, was the best I knew at the time
• Don’t let it get you down - you will get it right 9 times out of 10 if not more
• Early vs. Late events
• Early events: aaaaahhhh everything’s on fire, 6 teams can’t drive or pass inspection
• Simple problems, higher pressure
• Get teams running and move on to the next
• Have a team help once you get them moving
• Late events: problems are more advanced and sometimes just nice-to-haves
• Helping teams with vision or improving their autos
• Not always as much immediate time pressure in quals

Questions?