What is the iterator and how does it build an ecosystem in Go?
GopherDay Taiwan 2024
The Go gopher was designed by Renée French.
The gopher stickers was made by Takuya Ueda.
Licensed under the Creative Commons 3.0 Attributions license.
Hokkaido
@tenntenn
UEDA Takuya
Software Engineer / Knowledge Work Inc.
Google Developers Expert (GDE) in Go category
A board member of Gophers Japan
Go Conference Organizer
Contents
What’s the iterator?
The changes of for statement in Go1.22 and Go1.23
range over integers and functions
Range expression | 1st value | 2nd value |
array or slice a [n]E, *[n]E, or []E | index i int | a[i] E |
string s string type | index i int | rune |
map m map[K]V | key k K | m[k] V |
channel c chan E, <-chan E | element e E | |
integer n integer type | index i int | |
function, 0 values f func(func()bool) | | |
function, 1 value f func(func(V)bool) | value v V | |
function, 2 values f func(func(K, V)bool) | key k K | v V |
What is the iterator in Go?
A
B
E
C
D
A
B
E
C
D
The iterator provides sequential access to arbitrary data structures via a function (seq function).
seq()
for node := range seq
The seq and yield functions
func seq1(yield func(int) bool)) {
fmt.Println("in seq1")
}
for v := range seq1 {
fmt.Println("in loop")
}
func seq2(yield func(int) bool)) {
yield(100)
yield(200)
}
for v := range seq2 {
fmt.Println("in loop:", v)
}
$ go run .
in seq1
$ go run .
in loop: 100
in loop: 200
The seq function is called function once per a range statement.
The yield function passes values to the for range iterations.
Example: Depth First Search
for v := range seq
func seq(yield func(Node) bool))
yield( )
A
Iteration-1
A
B
C
D
E
v :
A
yield( )
B
Iteration-2
A
B
C
D
E
v :
B
yield( )
E
Iteration-5
A
B
C
D
E
v :
E
Breaking an iteration
func seq(yield func(int) bool)) {
fmt.Println(yield(100))
}
for v := range seq {
fmt.Println("in loop")
break
}
$ go run .
in loop
false
If a iteration will be breaked, the yield function returns false.
Creating a seq function dynamically
func all(ns []int) func(yield func(int) bool)) {
return func(yield func(int) bool)) {
for _, v := range ns {
if !yield(v) {
break
}
}
}
}
ns := []int{10, 20, 30}
for v := range all(ns) {
fmt.Println(v)
}
$ go run .
10
20
30
The created function captures an int slice via all function.
iter.Seq[V] and iter.Seq2[K,V]
type Seq[V any] func(yield func(V) bool)
func All[V any](s []V) iter.Seq2[int, V] {
return func(yield func(int, V) bool)) {
for i, v := range s {
if !yield(i, v) { break }
}
}
}
func Values[K, V any](seq2 iter.Seq2[K, V]) iter.Seq[V] {
return func(yield func(V) bool)) {
for _, v := range seq2 {
if !yield(v) { break }
}
}
}
type Seq2[K, V any] func(yield func(K, V) bool)
ns := []int{10, 20, 30}
for v := range Values(All(ns)) {
fmt.Println(v)
}
$ go run .
10
20
30
Live Coding Time!
The style of iterator with a function
Push style function (Go’s style)
Pull style function
value
func
func seq(v any) bool { }
The values are passed to a function as parameters.
func
value
func next() (any, bool) { }
The values are returned from a function.
iter.Pull() and iter.Pull2()
func Pair[V any](seq iter.Seq[V]) iter.Seq2[V, V] {
return func(yield func(V, V) bool)) {
next, stop := iter.Pull(seq)
defer stop()
for {
v1, _ := next(); v2, ok := next()
if !ok || !yield(v1, v2) { break }
}
}
}
ns := []int{10, 20, 30, 40}
for v1, v2 := range Pair(All(ns)) {
fmt.Println(v1, v2)
}
$ go run .
10 20
30 40
iter.Pull/Pull2 convert an iterator (seq function) to a pull style function.
Live Coding Time!
What's new with the iterator in Go1.23+?
Milestones of the iterator
Go1.23
Go1.24+
These milestones include tenntenn’s prediction.
The milestones may be changed and the predictions may be wrong.
✔ range over func (merged)
✔ iter.Seq/Seq2 (merged)
✔ iter.Pull/Pull2 (merged)
✔ slices: add iterator-related functions (merged)
✔ maps: add iterator-related functions (merged)
slices: add iterator-related functions
// All returns an iterator over index-value pairs in the slice.
func All[Slice ~[]Elem, Elem any](s Slice) iter.Seq2[int, Elem]
// Backward returns an iterator over index-value pairs in the slice, traversing it backward.
func Backward[Slice ~[]Elem, Elem any](s Slice) iter.Seq2[int, Elem]
// Values returns an iterator over the values in the slice, starting with s[0].
func Values[Slice ~[]Elem, Elem any](s Slice) iter.Seq[Elem]
// AppendSeq appends the values from seq to the slice and returns the extended slice.
func AppendSeq[Slice ~[]Elem, Elem any](x Slice, seq iter.Seq[Elem]) Slice
// Collect collects values from seq into a new slice and returns it.
func Collect[Elem any](seq iter.Seq[Elem]) []Elem
// Sorted collects values from seq into a new slice, sorts the slice, and returns it.
func Sorted[Elem comparable](seq iter.Seq[Elem]) []Elem
// SortedFunc collects values from seq into a new slice, sorts the slice, and returns it.
func SortedFunc[Elem any](seq iter.Seq[Elem], cmp func(Elem, Elem) int) []Elem
// SortedStableFunc collects values from seq into a new slice, sorts the slice using a stable sort, and returns it.
func SortedStableFunc[Elem any](seq iter.Seq[Elem], cmp func(Elem, Elem) int) []Elem
maps: add iterator-related functions
// All returns an iterator over key-value pairs from m.
func All[Map ~map[K]V, K comparable, V any](m Map) iter.Seq2[K, V]
// Collect collects key-value pairs from seq into a new map and returns it.
func Collect[K comparable, V any](seq iter.Seq2[K, V]) map[K]V
// Insert adds the key-value pairs from seq to m.
func Insert[Map ~map[K]V, K comparable, V any](m Map, seq iter.Seq2[K, V])
// Keys returns an iterator over keys in m.
func Keys[Map ~map[K]V, K comparable, V any](m Map) iter.Seq[K]
// Values returns an iterator over values in m.
func Values[Map ~map[K]V, K comparable, V any](m Map) iter.Seq[V]
x/exp/xiter: new package with iterator adapters
errors: add All and AllAs iterators
errors.All
err1 := errors.New("error1")
err2 := errors.New("error2")
err3 := fmt.Errorf("wrap: %w", err1)
err4 := fmt.Errorf("joined: %w, %w", err2, err3)
for err := errors.All(err4) {
fmt.Println(err)
}
$ go run .
joined: error2, wrap: error1
error2
wrap: erro1
error1
err4
err2
err3
err1
errors.AllAs
type MyError { index int; error}
err1 := MyError{ index:0; error:errors.New("error1") }
err2 := MyError{ index:1; error:errors.New("error2") }
err3 := fmt.Errorf("joined: %w, %w", err1, err2)
for err := errors.AllAs[MyError](err3) {
fmt.Println(err.index, err.error)
}
$ go run .
0 error1
1 error2
How does the iterator build an ecosystem in Go?
Histories of iterator
Useful abstractions in Go
Ecosystem of the iterator
slices
maps
graphs
iter.Seq
iter.Seq2
xiter.Map
xiter.Filter
slices
maps
graphs
iter.Seq
iter.Seq2
Data structures
Data structures
iterators
iterators
Use cases of the iterator
Thank you!