Awaiting The Future
Paul “LeoNerd” Evans (PEVANS)
Overview
A bit about Futures
Not a full lesson, just a quick introduction / reminder
A bit about syntax from other languages
A look at how we can do this in Perl
Futures
Useful building block
Asynchronous, single thread
Asynchronous Programming: Futures� - YAPC::EU 2014
Futures�Synchronous Programming
use List::Util qw( sum );�use LWP::UserAgent;��my $ua = LWP::UserAgent->new;��my $r1 = $ua->get("http://example.com/1");
�my $r2 = $ua->get("http://example.com/2");
�my $r3 = $ua->get("http://example.com/3");
�print sum( map { $_->content_length } $r1, $r2, $r3 );
Futures�Synchronous Programming
Functions "return" a result
Wait until the result arrives
Waiting for IO leads to "dead time"
Futures�Asynchronous Callbacks
use Hypothetical::HTTP::Callbacks qw( HTTP_GET );��HTTP_GET("http://example.com/1", sub {� my ( $response ) = @_;� print $response->content_length;�});
...
use List::Util qw( all sum );�use Hypothetical::HTTP::Callbacks qw( HTTP_GET );��my %lengths;�foreach my $page (qw( 1 2 3 )) {� $lengths{$page} = undef;� HTTP_GET("http://example.com/$page", sub {� my ( $response ) = @_;� $lengths{$page} = $response->content_length;� return unless all { defined $_ } values %lengths;�� print sum( values %responses );� });�}
Futures�Asynchronous Callbacks
...��my %lengths;�foreach my $page (qw( 1 2 3 )) {� $lengths{$page} = undef;� HTTP_GET("http://example.com/$page", sub {� my ( $response ) = @_;� $lengths{$page} = $response->content_length;� return unless all { defined $_ } values %lengths;�� HTTP_POST("http://example.com/total", sub {� my ( $response ) = @_;� print "Result was " . $response->code . "\n";� });� });�}
Futures�Asynchronous Callbacks
Divergence is easy
Convergence is hard
Sequencing looks messy
Futures�A First-Class "Next State"
Return a value representing an operation
Later that value can receive the result
Value acts as a proxy to ongoing operation
Convergence and sequencing are operations
They too can be represented by Futures
Futures�Asynchronous With Futures
use Hypothetical::HTTP::Futures qw( HTTP_GET );��my $f = HTTP_GET("http://example.com/1");
�my $response = $f->get;
�print $response->content_length;
Futures�Asynchronous With Futures
use Hypothetical::HTTP::Futures qw( HTTP_GET );��my $f = Future->needs_all(� HTTP_GET("http://example.com/1"),
HTTP_GET("http://example.com/2"),
HTTP_GET("http://example.com/3"),�);��my @responses = $f->get;
�print sum( map { $_->content_length } @responses );
Futures�An Inside-Out Callback
Nothing "magical" about a Future
It doesn't know about event loops, etc...
It's just an inside-out callback
A container of callback CODErefs
Stores a result value for later
Futures�Operations On Futures
Turn a Future into another one
my $f = HTTP_GET("http://example.com/page")� ->then( sub {� my ( $page ) = @_;� return HTTP_GET( $page->style_sheet_uri );� });��my $css = $f->get;
Futures�Operations On Futures
"Force" it with a ->get call
(some languages call this "force")
A blocking wait
Rude outside of toplevel program
#!/usr/bin/perl�use My::Modulino;��my $f = My::Modulino->run( @ARGV );��$f->get;
Futures�Operations On Futures
Use some sort of event loop
Relies on the event-style ->on_done callback
Futures�Asynchronous With Futures
Sequencing with lexical state is still awkward
Lexical variables can't "escape" ->then blocks
my $f = HTTP_GET("http://example.com/page")� ->then( sub {� my ( $page ) = @_;� return HTTP_GET( $page->style_sheet_uri );� })� ->then( sub {� my ( $css ) = @_;� print "Got the stylesheet for " . $page->title;� });
Futures�Asynchronous With Futures
my $page;��my $f = HTTP_GET("http://example.com/page")� ->then( sub {� ( $page ) = @_;� return HTTP_GET( $page->style_sheet_uri );� })� ->then( sub {� my ( $css ) = @_;� print "Got the stylesheet for " . $page->title;� });
The async/await syntax
Similar syntax across three other languages
Rust is thinking about it...
The async/await syntax�C# (5.0)
A type called Task, or generic Task<T>
Asynchronous programming� https://docs.microsoft.com/en-us/dotnet/csharp/async
public async Task<int> GetDotNetCountAsync()�{� var html = await client.DownloadStringAsync(� "http://dotnetfoundation.org");�� return Regex.Matches(html, ".NET").Count;�}
The async/await syntax�JavaScript (ES6)
A type called Promise
async function� https://developer.mozilla.org/� en-US/docs/Web/JavaScript/Reference/Statements/async_function
async function add1(x) {� var a = resolveAfter2Seconds(20);� var b = resolveAfter2Seconds(30);� return x + await a + await b;�}��add1(10).then(v => console.log(v));
The async/await syntax�Python 3
async def compute(x, y):
print("Compute %s + %s ..." % (x, y))� await asyncio.sleep(1.0)� return x + y
async def print_sum(x, y):
result = await compute(x, y)
print("%s + %s = %s" % (x, y, result))
The async/await syntax
async says "this function returns a Future"
allows the use of await expressions
await gets the result of a Future
by waiting if necessary, knowing it can
The async/await syntax
Works on a per-function basis
Suspend/resume a running function
Builds on top of existing Future-like type
Custom Syntax Modules
Perl 5.14 gave us the Keyword Hook
Allows XS code to provide new keywords
Other XS code can implement new semantics
Custom Keyword Plugins� - LPW 2016
Future::AsyncAwait
use Future::AsyncAwait;�use Hypothetical::HTTP::Futures qw( HTTP_GET );��async sub main�{� my @responses = await Future->needs_all(� HTTP_GET("http://example.com/1"),
HTTP_GET("http://example.com/2"),
HTTP_GET("http://example.com/3"),� );�� print sum( map { $_->content_length } @responses );�}��main()->get;
Future::AsyncAwait�Current Abilities
Lexical variables in scope are preserved
async sub main�{� my $page = await HTTP_GET("http://example.com/page");�� my $css = await HTTP_GET($page->style_sheet_uri);�� print "Got the stylesheet for " . $page->title;�}��main()->get;
Future::AsyncAwait�Current Abilities
await in the condition or body of an if block
async sub main�{� my $page = await HTTP_GET("http://example.com/page");�� if( my $uri = $page->style_sheet_uri ) {� my $css = await HTTP_GET($uri);� print "Got the stylesheet for " . $page->title;� }� ...�}��main()->get;
Future::AsyncAwait�Current Abilities
await in the condition or body of an while loop
async sub main�{� my $page = await HTTP_GET("http://example.com/page");�� while( $page->is_redirect ) {� $page = await HTTP_GET($page->redirect_uri);� }� ...�}��main()->get;
Future::AsyncAwait�Advantages
Invisible from the outside
Any async sub is just a function that returns a Future
Invisible on the inside
Can await on any expression that represents a Future
Incrementally applicable to codebase
Future::AsyncAwait�Advantages
Errors come from the right place
caller() sees the right function name
Implementation only has to understand `perl`
not the entire C state behind it
Future::AsyncAwait�Current Limitations
Can't handle await inside foreach loop body
Package (our) variables in scope confuse it
local is unhandled
Questions remain on the semantics required
Requires perl 5.24+
Mostly because of the context stack rework
Future::AsyncAwait�Implementation Details
Mostly generic CV suspend/resume logic
OP_AWAIT
Small Future-adapting wrapper
Small custom keyword parser hook
Future::AsyncAwait�Implementation Details
To suspend:
To resume:
Future::AsyncAwait�Where Next?
Implement unhandled foreach contexts
Implement correct handling of our vars
Support older perl versions
Ideally back as far as 5.14
Future::AsyncAwait�Where Next?
Consider the semantics of local
Does it warrant a new keyword?
our $DEBUG;��async sub main�{� local $DEBUG = 1;� await first();� print "Waiting for second\n" if $DEBUG;� await second(); �}
Future::AsyncAwait�Where Next?
Mixed-module testing
with Syntax::Keyword::Try
Future::AsyncAwait accidentally fell out of Syntax::Keyword::Try anyway
with other pad-walkers
with other runtime semantics
See Also
See Also�Other Talks
Asynchronous Programming with Futures:
YAPC::EU 2014� https://youtu.be/u9dZgFM6FtE� (slides)
Custom Keyword Plugins:
LPW 2016� (slides)
See Also�Blog Posts
“Futures advent calendar 2013”:
http://leonerds-code.blogspot.co.uk/� 2013/12/futures-advent-day-1.html
Thank You
Questions?