1 of 13

MikroTik Case Study for AS48635

Best Practices implementation using MikroTik

By Daryll Swer

March 2023

​

​

2 of 13

Preface

  • I will use a combination of this slide and a show-and-tell visual representation of the actual router configuration in real-time

3 of 13

Some basics on MikroTik

  • Different hardware models have different ASIC/Hardware design
    • The same model can have different revisions
  • Unlike other vendors MikroTik config is heavily hardware dependent
  • Hardware offloading and/or FastPath/FastForward/FastTrack will only work on config that complies with the ASIC/hardware design of a specific model
    • Layer ⅔ misconfiguration are common
  • Generally known to be buggy and nicknamed BugTik in the network engineering community
    • A one time netinstall of RouterOSv7 is recommended for clean slate

4 of 13

ASIC/Hardware Difference examples

  • RB1100x4 hardware (AMS01-02, 04-05, HAA01)

5 of 13

ASIC/Hardware Difference examples

  • CCR1009-8G-1S-1S+ hardware (AMS03)

6 of 13

ASIC/Hardware Difference examples

  • CCR2116-12G-4S+

7 of 13

ASIC/Hardware Differences’ Impact on config

  • Number of switch chips affects number of bridges
  • Generally speaking (single switch chip or zero switch chip) only a single bridge must exist
  • Irregular ASIC design of switched chip ports and non switched chip ports leads to convoluted config
  • MikroTik resolved this issue (almost) completely starting with their new CCR2k router and CRS3/5ks L3 switch models

8 of 13

Config changes Overview

  • Migrate L2/L3 to bridge config where it varies depending on hardware model – on-site
  • Clean-up invalid/outdated legacy config
    • Such as invalid static routes pointing to an upstream provider from years ago that no longer exists in the network
  • Implement best practices for router (host) config as per vendor guidelines and BCPs/RFCs
  • Implement a customised firewall config that’s RFC compliant and adapts to our specific needs for ACLs and traffic flow

9 of 13

Config changes deep dive

  • /system routerboard settings set auto-upgrade=yes
  • Enabling rp-filter loose mode, TCP Syn cookies, jumbo frames (L2)
  • Correctly configure IPv6 RAs to send RAs only for SLAAC enabled interfaces
    • This reduced BUM traffic
  • Create interface lists based on traffic flow logic specific to a given router
    • Use for simplified firewall config

10 of 13

Config changes deep dive

  • Set L2 MTU to max supported to minimise future changes and MTU profiles in the ASIC
  • Change conn_track values to match with traditional Linux values
  • Routing loops with RFC6890 space eliminated with static aggregated route to blackhole for the entire 6890 space (IPv4 and IPv6)
  • Disable unused services via IP>Services

11 of 13

Config changes deep dive

  • In RouterOS v7, we need to configure BGP input/output CPU affinity as per the underlying number of cores in the CPU
  • There are some minor left-over config that will not be copied over to the new CCR boxes to keep config as simple and as clean as possible
  • Possibly enable FQ_Codel on all physical ports for current-gen QoS/QoE to minimise bufferbloat (minimal noticeable improvements unless high bandwidth utilisation)
    • Refer to Dave Taht’s work on bufferbloat reduction for details
    • Should perhaps be enabled network-wide in the future

12 of 13

BUM Traffic reduction

13 of 13

Sources