Crazy Fun Experiments with PHP
(Not for Production)


Dutch PHP Conference 2019

lots of information ▶

About This Talk

  • 🚀 Lots of information, quickly

This talk contains a lot of information, and I’ll be going through it quickly to fit it all in.
Averages out to a slide transition every eighteen and a half seconds.

Slides, speaker notes, download ▶

About This Talk

  • 🚀 Lots of information, quickly
  • 📝 Slides, including speaker notes, are available to download

The slides, including speaker notes, are available to download if you want to go through it again. Which you probably will need to.

In hallway to discuss ▶

About This Talk

  • 🚀 Lots of information, quickly
  • 📝 Slides, including speaker notes, are available to download
  • 💬 I’ll be outside in the hallway during the next talk for anyone who wants to discuss this talk

On top of trying to fit in answering questions at the end which probably won’t happen, I’ll be outside this room during lunch to have discussions with anyone who’s interested.

Technical talk, don’t have to understand ▶

About This Talk

  • 🚀 Lots of information, quickly
  • 📝 Slides, including speaker notes, are available to download
  • 💬 I’ll be outside in the hallway during the next talk for anyone who wants to discuss this talk
  • 🤓 This is a technical talk, but don’t worry about keeping up with every code example.

This is a technical talk, don’t worry about keeping up with every code example presented. There are a lot of them, the overall idea is more important.

Go huh? What? ▶

About This Talk

  • 🚀 Lots of information, quickly
  • 📝 Slides, including speaker notes, are available to download
  • 💬 I’ll be outside in the hallway during the next talk for anyone who wants to discuss this talk
  • 🤓 This is a technical talk, but don’t worry about keeping up with every code example.

what?

huh?

This talk is designed to present you with unfamiliar ideas and concepts

More importantly, I wonder, what if ▶

About This Talk

  • 🚀 Lots of information, quickly
  • 📝 Slides, including speaker notes, are available to download
  • 💬 I’ll be outside in the hallway during the next talk for anyone who wants to discuss this talk
  • 🤓 This is a technical talk, but don’t worry about keeping up with every code example.

what?

huh?

I wonder...

what if?

Most importantly, even if it’s for a split second, I want this talk to make you go “I wonder…” or “what if…?

Performance ▶

PERFORMANCE

For remainder of talk, imagine that we don’t care about performance in our applications.

Streams

Streams

Streams (Computer Science) ▶

Computer Science

A sequence of data elements made available over time.

Can have finite size, or be continuous.

In computer science, a stream is a sequence of data elements made available over time. Think of it as items on a conveyor belt that are processed as they arrive - most streams, like conveyor belts, end once there are no more items but can run continuously if need be.

Streams (in PHP) ▶

Computer Science

A sequence of data elements made available over time.

Can have finite size, or be continuous.

PHP Streams

A way of generalizing file, network and other operations which share a common set of functions.

A stream is a resource object that can be read from or written to in a linear fashion.

If you write PHP applications deal with input/output of data, you use streams. They’re core function in PHP and pretty unavoidable for any developer.

Streams in PHP way of generalizing file, network, data compression & other operations which share common set of functions and uses: stream is resource object which can be read from written to in linear fashion.

Use streams in Guzzle, filesystem, POST, etc ▶

You’ve Used Streams

  • Guzzle
  • Sockets
  • Accessed the filesystem
  • Dealt with a $_POST request

If you’ve used Guzzle, used sockets, accessed the filesystem, or even dealt with a $_POST request, you’ve used streams. PHP has a low barrier-of-entry so the vast majority of stream handling is abstracted away from the developer.

For example, file_get_contents

file_get_contents(‘ hello.txt’);

for example, file_get_contents(‘hello.txt’) really means…

file:// wrapper ▶

file_get_contents(‘file://hello.txt’);

Fetch the requested URI using the file:// protocol

Supported protocols ▶

Supported Protocols (Stream Wrappers)

Native PHP

  • file
  • http
  • ftp
  • php
  • zlib
  • bzip2
  • data
  • glob
  • phar

Known as stream wrappers, native PHP has the following to provide support for protocols.

Each wrapper may support multiple protocols, such as http

Protocols in PHP extensions ▶

Supported Protocols (Stream Wrappers)

Native PHP

  • file
  • http
  • ftp
  • php
  • zlib
  • bzip2
  • data
  • glob
  • phar

Enabled through PHP extensions

  • zip
  • ssh2
  • rar
  • ogg
  • expect

The following protocols are available if their associated extensions are installed: zip, ssh2, rar, ogg, and expect.

s3:// userland wrapper ▶

s3://bucket/key

Who has come across the s3:// protocol while using the AWS SDK before?
PHP lets us define our own stream wrappers in userland PHP.

Esoteric language (Brainf*ck) ▶

Esoteric Language

Brainf*ck

Introducing Brainf*ck!
Obviously not exactly the best name to use at an international conference but it’s perhaps the world’s most well known esoteric language.

hello.bf

Esoteric Language

Brainf*ck

++++++++[>++++[>++>+++>+++>+<<<<-]>+>+>->>+[<]<-]>>.>---.+++++++..+++.>>.<-.<.+++.------.--------.>>+.>++.

hello.bf

Let’s assume we have a script hello.bf - if we require that file, the contents will get dumped byte-for-byte because the PHP interpreter doesn’t find any PHP code.

Write application in BF ▶

require ‘ /app/hello.bf’;

If, for some arcane, horrendous reason, we wanted to write parts of our application in BF, and require it in our application

file:// equivalent ▶

require ‘file:///app/hello.bf’;

Which is equivalent to using the file:// stream wrapper. This is no good.

Custom/userland wrapper ▶

require ‘ bf:///app/hello.bf’;

We can substitute this for a custom stream wrapper in userland PHP to manipulate the input before including the result

StreamWrapper class boilerplate ▶

stream_open·stream_stat·stream_read·stream_eof·stream_close

class BfStreamWrapper

{

/** @var resource $context */

public $context;

public static function register(string $filterName): void

{

stream_wrapper_register($filterName, static::class);

}

// ...









}

Here we have the boilerplate for a stream wrapper, the class definition and a static method to register it with PHP.
Stream wrappers don’t extend or implement anything in PHP core, but we need to implement a minimum of 5 methods to process an incoming stream.

stream_open()

stream_open·stream_stat·stream_read·stream_eof·stream_close

class BfStreamWrapper

{

// ...


private $uri;



public function stream_open(string $path, string $mode, int $options, &$opened_path): bool

{

$realpath = $this->removeProtocolFromPath($path, static::WRAPPER_PROTOCOL);

if (file_exists('file://' . $realpath)) {

$this->uri = $opened_path = $realpath;

return true;

}

return false;

}


}

The first is stream_open(), we’re just checking that the file specified exists and taking note of its path.

stream_stat()

stream_open·stream_stat·stream_read·stream_eof·stream_close

class BfStreamWrapper

{

// ...

private $uri;


public function stream_stat(): array

{

// Should probably return something useful here, but not

// a hard requirement.

return [];

}





}

stream_stat() is something that’s called each time to collect information, it’s recommended but apart from having to exist it isn’t exactly a hard requirement.

stream_read()

stream_open·stream_stat·stream_read·stream_eof·stream_close

class BfStreamWrapper

{

// ...

private $uri;

private $output;

private $pointer = 0;

private $eof = false;

public function stream_read(int $readNumBytes)

{

$this->execute();

$remainingBytes = strlen($this->output) - $this->pointer;

if ($remainingBytes > 0) {

$buffer = substr($this->output, $this->pointer, $readNumBytes);

$this->pointer += $readNumBytes;

return $buffer;

}

$this->eof = true;

return false;

}

}

Streams are read from written to in chunks, meaning stream_read() invoked multiple times. Method is more complicated than the rest because needs to keep track of how much data flowed out.

Simplified, first line is BF script getting executed every line after is dealing with returning the buffered script output in chunks as they’re requested by stream user.

stream_eof()

stream_open·stream_stat·stream_read·stream_eof·stream_close

class BfStreamWrapper

{

// ...

private $uri;

private $output;

private $pointer = 0;

private $eof = false;

public function stream_eof(): bool

{

return $this->eof;

}







}

The stream_eof() method returns a boolean value indicating whether the previous stream_read() method has reached the end of the stream.

stream_close()

stream_open·stream_stat·stream_read·stream_eof·stream_close

class BfStreamWrapper

{

// ...

private $uri;

private $output;

private $pointer = 0;

private $eof = false;

public function stream_close(): bool

{

// Clear internal buffers.

$this->uri = null;

$this->output = null;

$this->pointer = 0;

$this->eof = false;

return true;

}


}

stream_close() is just cleaning up the internal state - in other protocols this might be where you close a connection or delete a lock file.

execute()

stream_open·stream_stat·stream_read·stream_eof·stream_close

class BfStreamWrapper

{

// ...

private $input = [];

private $output;

private function execute(): void

{

if (is_string($output)) {

return;

}

$result = (new \Brainfuck\Language)->run(

file_get_contents('file://' . $this->uri),

$this->input

);

$this->output = implode('', array_map(function (int $ord): string {

// Brainf*ck returns result in bytes (8-bit integers). Convert to ASCII.

return chr($ord);

}, $result));

}

}

Finally, for completeness, the powerhouse of stream wrapper: the method that executes BF scripts. In this example I’m using Anthony Ferrara’s (@ircmaxell) library.

We take the contents of the BF script, pump it through the language runtime, and save the result as a string in the $output class property.

Input? ▶

Input?

However, requiring a script using our new stream wrapper doesn’t give us a change to provide the script with any input, which is kind of important for some scripts.

stream_write()

stream_write

class BfStreamWrapper

{

// ...

private $input = [];

public function stream_write(string $data): int

{

// No point in recording input if script has already been executed.

if (is_string($this->output)) {

return 0;

}

$count = 0;

foreach (str_split($data) as $chr) {

// Brainf*ck takes bytes (8-bit integers) as input.

$this->input[] = ord($chr);

$count++;

}

return $count;

}

}

We can accept an input string, and save it in a class property to use when we execute the script.
Unfortunately in our example this means that we can only accept input before we execute our script, which is whenever we attempt to read the output.

Using it (index.php) ▶

index.php

BfStreamWrapper::register('bf');

$stream = 'bf:///app/hello.bf';

$handle = fopen($stream, ‘r+’);

fwrite($handle, 'script input');

$output = stream_get_contents($handle);

Now it’s time to use it!
After registering our stream filter, we can open up a stream handle, pump in some input, and get the output of our script!

Notice how if our script needs input, we can’t require the script like we would a PHP file.

Stream Filters ▶

Stream Filters

Next up in the PHP streams arsenal is stream filters. They’re simpler than wrappers: just a method to modify data as it passes through - this does mean that it cannot deal with both input and output so this is a perfect time to introduce source code transformation!

Stream filter boilerplate ▶

php_stream_filter Implementation

class BfStreamFilter extends \php_user_filter

{

public static function register(string $filterName): void

{

stream_filter_register($filterName, self::class);

}

// ...

}

Here we have the boilerplate for a stream filter, the class definition and a static method to register it with PHP.
Unlike wrappers, stream filters extend a core PHP class and only need to implement one method.

filter() method ▶

php_stream_filter Implementation

public function filter($in, $out, &$consumed, $closing): int

{

while ($bucket = stream_bucket_make_writeable($in)) {

$this->input .= $bucket->data;

}

if ($closing || feof($this->stream)) {

$consumed = strlen($this->input);

$bucket = stream_bucket_new(

$this->stream,

// Has to be static because userland doesn't control instantiation.

static::$twig->render('bf_closure.twig.php', [

'bf_php_value' => var_export($this->input, true),

])

);

stream_bucket_append($out, $bucket);

return \PSFS_PASS_ON;

}

return \PSFS_FEED_ME;

}

Again, streams are processed in chunks so the vast majority of this method is dealing with pulling in the contents of the stream, possibly over multiple invocations…

Highlight input/output ▶

php_stream_filter Implementation

public function filter($in, $out, &$consumed, $closing): int

{

while ($bucket = stream_bucket_make_writeable($in)) {

$this->input .= $bucket->data;

}

if ($closing || feof($this->stream)) {

$consumed = strlen($this->input);

$bucket = stream_bucket_new(

$this->stream,

// Has to be static because userland doesn't control instantiation.

static::$twig->render('bf_closure.twig.php', [

'bf_php_value' => var_export($this->input, true),

])

);

stream_bucket_append($out, $bucket);

return \PSFS_PASS_ON;

}

return \PSFS_FEED_ME;

}

But it still boils down to bringing data in, and outputting after we’ve processed it.

In this example we’re generating valid PHP code by using a code template.

bf_closure.twig.php

bf_closure.php.twig

<?php declare(strict_types=1);

return function (array $input = []): string {

$result = (new \Brainfuck\Language)->run(

,

$input

);

return implode('', array_map(

function (int $ord): string {

// BF returns result in bytes (8-bit

// integers). Convert to ASCII.

return chr($ord);

},

$result

));

};

{% Brainf*ck Closure Code Template %}



{{ bf_php_value }}

Almost every time I’ve had to output XML from an application, I’ve used Twig rather than deal with encoding data structures. Why not do it for PHP, too?

Using it (index.php) ▶

index.php

BfStreamFilter::register('bf');

$handle = fopen('/app/hello.bf', 'r+');

stream_filter_append($handle, 'bf');

$closureCode = stream_get_contents($handle);

$closure = eval('?>' . $closureCode);

$output = $closure($input);

This is similar to before.

We register our stream filter, except this time we attach our filter to an already existing stream handle.

When we read the contents of the stream we get back valid PHP code that we can execute!

php://filter

php://filter

Attaching a filter to a stream handle and evaluating each time isn’t ideal - and most importantly we can’t attach a stream filter to a require statement.

That is where one of the most underrated features of PHP comes in - the php:// stream wrapper…
In particular the php://filter meta-wrapper.

/resource=

php://filter

/resource=<resource-name>

The resource identifier of the stream you’d like to filter.

The php://filter meta-wrapper allows us to act upon any other stream resource, while…

/read=

php://filter

/resource=<resource-name>


/read=<list,of,filter,names>

The resource identifier of the stream you’d like to filter.

Apply the specified filters when reading data from the stream.

Attaching any number of filters to manipulate the stream when reading it…

/write=

php://filter

/resource=<resource-name>


/read=<list,of,filter,names>


/write=<list,of,filter,names>

The resource identifier of the stream you’d like to filter.

Apply the specified filters when reading data from the stream.

Apply the specified filters when writing data to the stream.

As well as when writing to it.

Example (php://filter/read=/resource=) ▶

php://filter

BfStreamFilter::register('bf');

$script = '/app/hello.bf';

# php://filter/read=bf/resource=file:///app/hello.bf

$closure = require 'php://filter'

. '/read=bf'

. '/resource=file://' . $script;

$output = $closure($input);

Most importantly, this allows us to specify a resource and attach a filter to it all in one string which we can pass to require.

This one string saves us from having to open up a handle, attaching a filter, and evaluating the result of the stream.

Go! AOP ▶

Go! AOP
by Alexander Lisachenko

@ DPC2016, I saw Alexander Lisachenko talk about Go! AOP framework he created, which is main inspiration for this talk. I’ll give extremely brief idea of what project does, then we’ll delve straight into internals and figure out what’s going on.
Sidenote: due to the fact there is go the game, go the language, and go the framework, I’ll just refer to it as AOP.

Aspect-oriented programming ▶

Aspect-oriented Programming

  • Allows separation of cross-cutting concerns
  • Increases modularity
  • Adds additional behaviour to existing code

AOP is a framework for developing PHP applications using aspect-oriented programming: separating out logic when developing and joining it back together at runtime.

To keep this easier to digest I’ll use the same examples as the creator of this framework and I recommend you look up Lisachenko’s previous presentations on this subject.

createNewUser()

function createNewUser(string $email, string $password): UserInterface {

$user = new User($email, $password);

$this->entityManager->persist($user);

$this->entityManager->flush();

return $user;

}

Let’s say we have a method that creates a new user in our application.

As far as business logic goes, that’s all that’s required for creating a new user.

Real world ▶

function createNewUser(string $email, string $password): UserInterface {



$user = new User($email, $password);

$this->entityManager->persist($user);

$this->entityManager->flush();





return $user;

}

But in the real world things are never this simple:

Authorization ▶

function createNewUser(string $email, string $password): UserInterface {

if (!$this->security->isGranted(‘ROLE_ADMIN’)) {

throw new AccessDeniedException;

}

$user = new User($email, $password);

$this->entityManager->persist($user);

$this->entityManager->flush();





return $user;

}

We’ll need to the authorization of the currently logged-in user to make sure they’re allowed to create new users.

Log creation ▶

function createNewUser(string $email, string $password): UserInterface {

if (!$this->security->isGranted(‘ROLE_ADMIN’)) {

throw new AccessDeniedException;

}

$user = new User($email, $password);

$this->entityManager->persist($user);

$this->entityManager->flush();




$this->logger->info(‘User created successfully.’, [‘email’ => $email]);

return $user;

}

We should log that a new user is being created.

Emit event ▶

function createNewUser(string $email, string $password): UserInterface {

if (!$this->security->isGranted(‘ROLE_ADMIN’)) {

throw new AccessDeniedException;

}

$user = new User($email, $password);

$this->entityManager->persist($user);

$this->entityManager->flush();



$this->eventDispatcher->dispatch(new UserCreatedEvent($email));

$this->logger->info(‘User created successfully.’, [‘email’ => $email]);

return $user;

}

We need to emit an event so that an email can be sent asynchronously to the new user informing them that their account is ready to use.

Exception handling ▶

function createNewUser(string $email, string $password): UserInterface {

if (!$this->security->isGranted(‘ROLE_ADMIN’)) {

throw new AccessDeniedException;

}

try {

$user = new User($email, $password);

$this->entityManager->persist($user);

$this->entityManager->flush();

} catch (ORMException $e) {

$this->logger->error(‘Could not persist new user.’, [‘email’ => $email]);

throw $e;

}

$this->eventDispatcher->dispatch(new UserCreatedEvent($email));

$this->logger->info(‘User created successfully.’, [‘email’ => $email]);

return $user;

}

And we need exception handling in case something goes wrong.

Consider many different concerns ▶

function createNewUser(string $email, string $password): UserInterface {

if (!$this->security->isGranted(‘ROLE_ADMIN’)) {

throw new AccessDeniedException;

}

try {

$user = new User($email, $password);

$this->entityManager->persist($user);

$this->entityManager->flush();

} catch (ORMException $e) {

$this->logger->error(‘Could not persist new user.’, [‘email’ => $email]);

throw $e;

}

$this->eventDispatcher->dispatch(new UserCreatedEvent($email));

$this->logger->info(‘User created successfully.’, [‘email’ => $email]);

return $user;

}

While this example may not be unwieldy once everything has been factored in, it does demonstrate how even the simplest things need to consider many different cross-cutting concerns. More complex situations can get unwieldy very quickly.

Interrupting execution flow (pointcuts) ▶

function createNewUser(string $email, string $password): UserInterface {

$user = new User($email, $password);

$this->entityManager->persist($user);

$this->entityManager->flush();

return $user;

}

Aspect-oriented programming is about keeping your method simple and interrupting the execution flow of the application at specific points to add logic.
You keep logic separated, and inject each piece to where it needs to be.
These injected pieces of logic are called pointcuts.

First pointcut ▶

function createNewUser(string $email, string $password): UserInterface {

$user = new User($email, $password);

$this->entityManager->persist($user);

$this->entityManager->flush();

return $user

}

/** @Before(pointcut=”public UserService->*(*)”) */

function checkCurrentUserCanCreateUsers(MethodInvocation $method): void {

if (!$this->security->isGranted('ROLE_ADMIN')) {

throw new AccessDeniedException;

}

}

Our first pointcut is the logic for ensuring that the currently logged-in user is authorized to use the User service…

Happens @Before

function createNewUser(string $email, string $password): UserInterface {

$user = new User($email, $password);

$this->entityManager->persist($user);

$this->entityManager->flush();

return $user

}

/** @Before(pointcut=”public UserService->*(*)”) */

function checkCurrentUserCanCreateUsers(MethodInvocation $method): void {

if (!$this->security->isGranted('ROLE_ADMIN')) {

throw new AccessDeniedException;

}

}

… and this happens before any public method on the User service is called.

Next up: Error handling ▶

function createNewUser(string $email, string $password): UserInterface {

$user = new User($email, $password);

$this->entityManager->persist($user);

$this->entityManager->flush();

return $user

}

/** @Before(pointcut=”public UserService->*(*)”) */

function checkCurrentUserCanCreateUsers(MethodInvocation $method): void {

if (!$this->security->isGranted(‘ROLE_ADMIN’)) {

throw new AccessDeniedException;

}

}

/** @AfterThrow(pointcut=”public UserService->createNewUser(*)”) */

function handleNewUserDatabaseError(MethodInvocation $method): void {

$this->logger->error('Could not persist new user.', ['email' => $method->getArguments()[0]]);

throw $e;

}

Next up is our error handling logic…

Happens @AfterThrow

function createNewUser(string $email, string $password): UserInterface {

$user = new User($email, $password);

$this->entityManager->persist($user);

$this->entityManager->flush();

return $user

}

/** @Before(pointcut=”public UserService->*(*)”) */

function checkCurrentUserCanCreateUsers(MethodInvocation $method): void {

if (!$this->security->isGranted(‘ROLE_ADMIN’)) {

throw new AccessDeniedException;

}

}

/** @AfterThrow(pointcut=”public UserService->createNewUser(*)”) */

function handleNewUserDatabaseError(MethodInvocation $method): void {

$this->logger->error('Could not persist new user.', ['email' => $method->getArguments()[0]]);

throw $e;

}

And this happens after an exception is thrown from the original business logic.

Finally ▶

function createNewUser(string $email, string $password): UserInterface {

$user = new User($email, $password);

$this->entityManager->persist($user);

$this->entityManager->flush();

return $user

}

/** @Before(pointcut=”public UserService->*(*)”) */

function checkCurrentUserCanCreateUsers(MethodInvocation $method): void {

if (!$this->security->isGranted(‘ROLE_ADMIN’)) {

throw new AccessDeniedException;

}

}

/** @AfterThrow(pointcut=”public UserService->createNewUser(*)”) */

function handleNewUserDatabaseError(MethodInvocation $method): void {

$this->logger->error(‘Could not persist new user.’, [‘email’ => $method->getArguments()[0]]);

throw $e;

}

/** @After(pointcut=”public UserService->createNewUser(*)”) */

function onNewUserSuccessfullyCreated(MethodInvocation $method): void {

$email = $method->getArguments()[0];

$this->eventDispatcher->dispatch(new UserCreatedEvent($email));

$this->logger->info('User created successfully.', ['email' => $email]);

}

And finally we have the logic to emit an event and log the creation of the user…

Happens @After

function createNewUser(string $email, string $password): UserInterface {

$user = new User($email, $password);

$this->entityManager->persist($user);

$this->entityManager->flush();

return $user

}

/** @Before(pointcut=”public UserService->*(*)”) */

function checkCurrentUserCanCreateUsers(MethodInvocation $method): void {

if (!$this->security->isGranted(‘ROLE_ADMIN’)) {

throw new AccessDeniedException;

}

}

/** @AfterThrow(pointcut=”public UserService->createNewUser(*)”) */

function handleNewUserDatabaseError(MethodInvocation $method): void {

$this->logger->error(‘Could not persist new user.’, [‘email’ => $method->getArguments()[0]]);

throw $e;

}

/** @After(pointcut=”public UserService->createNewUser(*)”) */

function onNewUserSuccessfullyCreated(MethodInvocation $method): void {

$email = $method->getArguments()[0];

$this->eventDispatcher->dispatch(new UserCreatedEvent($email));

$this->logger->info('User created successfully.', ['email' => $email]);

}

… and this happens after the successful completion of the original business logic method.

Can decorate @Around

/** @Around(pointcut=”public UserService->createNewUser(*)”) */

function logTimeTakenToCreateUser(MethodInvocation $method): UserInterface {

$start = microtime(true);

$result = $method->proceed();


$duration = microtime(true) - $start;

$this->logger->log(sprintf('Creating a user took %f seconds', $duration));


// Modify $result before returning?
return $result;

}

We can even decorate the original method and make modifications to the result if we want to.

Source transformers ▶

Source Transformers

The majority of the AOP’s logic is in the form of source transformers.
By using AOP in your application, you’re entering the realm of meta-programming: your application transforms its own source code when including it.

Transformation process ▶

Go! AOP’s source transforming process:

So what is AOP doing inside that transformer?

Queries Composer ▶

Go! AOP’s source transforming process:

  • Go! AOP queries Composer for file location

AOP queries Composer’s class loader for the location of the file that should contain the class definition for MyClass

Generate metadata ▶

Go! AOP’s source transforming process:

  • Go! AOP queries Composer for file location
  • Generates metadata about the filter input (original source code) file
    Including the source code’s Abstract Syntax Tree (using nikic/php-parser).

It generates a load of metadata about the source code file, including its contents and the Abstract Syntax Tree generated from its contents by Nikita Popov’s pure-PHP language parser nikic/php-parser.

Passes through series of transformers ▶

Go! AOP’s source transforming process:

  • Go! AOP queries Composer for file location
  • Generates metadata about the filter input (original source code) file
    Including the source code’s Abstract Syntax Tree (using nikic/php-parser).
  • Pass metadata through a series of source transformers
    Each manipulating the AST to provide a particular feature, such as rewriting require’s.

It passes that metadata through a series of source transforming classes, each manipulating the AST to provide a particular feature.

Dump AST to PHP ▶

Go! AOP’s source transforming process:

  • Go! AOP queries Composer for file location
  • Generates metadata about the filter input (original source code) file
    Including the source code’s Abstract Syntax Tree (using nikic/php-parser).
  • Pass metadata through a series of source transformers
    Each manipulating the AST to provide a particular feature, such as rewriting require’s.
  • Dump final manipulated AST back into PHP code
    Think of this as the warm-up, result is saved to cache so subsequent requests skip source transformation stage.

Dump the final manipulated AST back into PHP code as a string.
It’s at this point that the final compiled source is saved to cache so that subsequent requests skip the expensive source transformation stage.

Output is streamed to PHP ▶

Go! AOP’s source transforming process:

  • Go! AOP queries Composer for file location
  • Generates metadata about the filter input (original source code) file
    Including the source code’s Abstract Syntax Tree (using nikic/php-parser).
  • Pass metadata through a series of source transformers
    Each manipulating the AST to provide a particular feature, such as rewriting require’s.
  • Dump final manipulated AST back into PHP code
    Think of this as the warm-up, result is saved to cache so subsequent requests skip source transformation stage.
  • Streams final PHP code string as the output of the filter
    To be executed by the PHP engine.

The final transformed code is returned to be executed by the PHP engine.

Automagically ▶

Automagically ✨

$userService = new UserService(/* dependencies */);

$user = $userService->createNewUser(

'admin@example.com',

'Password'

);

All of this happens while you’re writing PHP applications normally without worrying about loading classes or compiling into usable code.
As this talk is all about PHP streams, it may not come as a surprise that this main logic behind the AOP framework all happens from within a PHP Stream Filter

transformCodeFilter()

$executableCodeContents =

transformCodeFilter(

$sourceCodeFileOnDisk

);

A huge amount of complexity is held within that process, enough for an entirely different presentation, so for the sake of time constraints of this talk, we’ll simplify AOP’s source transforming process into a single function call.

From now on, we’ll use this function as an alias to implementing and registering a stream filter because, quite frankly, implementing a stream filter doesn’t fit on a slide easily!

Autoloader Overloading ▶

Autoloader Overloading

new MyClass???transformCodeFilter()

But if you try to create a new instance of MyClass, how does AOP manipulate Composer to pass the source code file through the transformCodeFilter() function before getting PHP to execute the contents?

Instantiate new MyClass

new MyClass;

You go to instantiate a new class.

Composer does its thing ▶

new MyClass;

Composer determines source file location:
/var/www/MyClass.php

Composer does its normal thing of figuring out where the class definition is located.

Hijacks Composer ▶

new MyClass;

Composer determines source file location:
/var/www/MyClass.php

Go! AOP suddenly says “I’d like to interject for a moment…

AOP hijacks the normal Composer logic…

Change URI to load ▶

new MyClass;

Composer determines source file location:
/var/www/MyClass.php

Go! AOP suddenly says “I’d like to interject for a moment…

The framework instructs PHP to instead load:
php://filter
/read=go.source.transforming.loader

/resource=file:///var/www/MyClass.php

… and tells PHP to load a new URI that uses the PHP stream wrapper…

Apply stream filter ▶

new MyClass;

Composer determines source file location:
/var/www/MyClass.php

Go! AOP suddenly says “I’d like to interject for a moment…

The framework instructs PHP to instead load:
php://filter
/read=
go.source.transforming.loader
/resource=file:///var/www/MyClass.php

PHP then applies the go.source.transforming.loader stream filter…

When reading contents of URI ▶

new MyClass;

Composer determines source file location:
/var/www/MyClass.php

Go! AOP suddenly says “I’d like to interject for a moment…

The framework instructs PHP to instead load:
php://filter
/read=
go.source.transforming.loader
/resource=file://
/var/www/MyClass.php

… when reading the contents of this URI …

From filesystem ▶

new MyClass;

Composer determines source file location:
/var/www/MyClass.php

Go! AOP suddenly says “I’d like to interject for a moment…

The framework instructs PHP to instead load:
php://filter
/read=go.source.transforming.loader

/resource=
file:///var/www/MyClass.php

… from the file system …

What sneaky magic? ▶

new MyClass;

Composer determines source file location:
/var/www/MyClass.php

Go! AOP suddenly says “I’d like to interject for a moment…

The framework instructs PHP to instead load:
php://filter
/read=go.source.transforming.loader

/resource=
file:///var/www/MyClass.php

Sneaky! 🐍

So what kind of sneaky magic is this?

Fetch autoloaders ▶

spl_autoload_functions();

Go! AOP

Hey, PHP! Give me all of the registered autoloaders please! 🙏

AOP will fetch all the currently loaded autoloaders using the spl_autoload_functions() function.

spl_autoload_functions() return array ▶

[ Composer\Autoload\ClassLoader, MyAppLoader ]

Go! AOP

Hey, PHP! Give me all of the registered autoloaders please! 🙏

spl_autoload_functions() will return an array of autoloaders.

Iterate ▶

[ Composer\Autoload\ClassLoader, MyAppLoader ]

Hmm, let’s take a closer look at those… 🔍

Go! AOP

Hey, PHP! Give me all of the registered autoloaders please! 🙏

AOP will iterate over each autoloader.

Decorate and replace ▶

[ Composer\Autoload\ClassLoader, MyAppLoader ]

[ AopComposerLoader(Composer\Autoload\ClassLoader), MyAppLoader ]

Go! AOP

Hey, PHP! Give me all of the registered autoloaders please! 🙏

Hmm, let’s take a closer look at those… 🔍

If AOP finds Composer in the list of autoloaders, it will decorate it and replace the original Composer autoloader it found

Pretend all along ▶

[ Composer\Autoload\ClassLoader, MyAppLoader ]

[ AopComposerLoader(Composer\Autoload\ClassLoader), MyAppLoader ]

Hot dang! I knew it!

Go! AOP

Hey, PHP! Give me all of the registered autoloaders please! 🙏

Hmm, let’s take a closer look at those… 🔍

Pretend it was there all along

Re-register ▶

[ Composer\Autoload\ClassLoader, MyAppLoader ]

[ AopComposerLoader(Composer\Autoload\ClassLoader), MyAppLoader ]

Okay, thanks PHP! You can have those back now! 😘

Go! AOP

Hey, PHP! Give me all of the registered autoloaders please! 🙏

Hmm, let’s take a closer look at those… 🔍

Hot dang! I knew it!

Then re-register the autoloaders with PHP

Looks legit ▶

[ Composer\Autoload\ClassLoader, MyAppLoader ]

[ AopComposerLoader(Composer\Autoload\ClassLoader), MyAppLoader ]

Sure, looks legit 🤷‍♀️

Go! AOP

PHP

Hey, PHP! Give me all of the registered autoloaders please! 🙏

Hmm, let’s take a closer look at those… 🔍

Hot dang! I knew it!

Okay, thanks PHP! You can have those back now! 😘

PHP accepts the divine wisdom of AOP

The Magic™ (Class Boilerplate) ▶

The Magic™

class AutoloaderOverloader

{

private $composer;

private $filterName;

protected function __construct(ComposerClassLoader $composer, string $filterName) {

$this->composer = $composer;

$this->filterName = $filterName

}










}

Our overloader. Since it decorates Composer we accept it as the first argument, and it’s always useful to know the name of the filter we should be applying.

loadClass() ▶

The Magic™

class AutoloaderOverloader

{

private $composer;

private $filterName;

protected function __construct(ComposerClassLoader $composer, string $filterName) {

$this->composer = $composer;

$this->filterName = $filterName

}

public function loadClass($class) {

$file = $this->composer->findFile($class);

$compiledFile = $this->cache->findCachedFile($file)

?: 'php://filter/read=' . $this->filterName . '/resource=file://' . $file;

include $compiledFile;

}



}

The autoloader part: all of the hard work is still done by Composer, all we do it rewrite the file path so that the php:// stream wrapper applies a filter when reading the file.

No file returned ▶

The Magic™

class AutoloaderOverloader

{

private $composer;

private $filterName;

protected function __construct(ComposerClassLoader $composer, string $filterName) {

$this->composer = $composer;

$this->filterName = $filterName

}

public function loadClass($class) {

$file = $this->composer->findFile($class);

$compiledFile = $this->cache->findCachedFile($file)

?: 'php://filter/read=' . $this->filterName . '/resource=file://' . $file;

include $compiledFile;

}



}

No file returned?
Code templates!

We’ve already established that we can generate code using templates, this would be the place to do it.

If that’s all you wanted to do we wouldn’t even need to bother with filters!

Static helper method (init) ▶

The Magic™

class AutoloaderOverloader

{

// ...

public static function init(string $filterName): void

{

$loaders = spl_autoload_functions();

foreach ($loaders as &$loader) {

// Unregister each loader, so they can be re-registered in the same order.

spl_autoload_unregister($loader);

if (is_array($loader) && $loader[0] instanceof ComposerClassLoader) {

// Replace Composer autoloader with our hijacking autoloader.

$loader[0] = new self($loader[0], $filterName);

}

}

// Processed all the loaders, re-register them with PHP in their original order.

foreach ($loaders as $loader) {

spl_autoload_register($loader);

}

}

}

Next, the overloader part, we add a static helper method to initialize our overloader.

Just like we described before in less technical terms, we…

Fetch registered autoloaders ▶

The Magic™

class AutoloaderOverloader

{

// ...

public static function init(string $filterName): void

{

$loaders = spl_autoload_functions();

foreach ($loaders as &$loader) {

// Unregister each loader, so they can be re-registered in the same order.

spl_autoload_unregister($loader);

if (is_array($loader) && $loader[0] instanceof ComposerClassLoader) {

// Replace Composer autoloader with our hijacking autoloader.

$loader[0] = new self($loader[0], $filterName);

}

}

// Processed all the loaders, re-register them with PHP in their original order.

foreach ($loaders as $loader) {

spl_autoload_register($loader);

}

}

}

Fetch the list of registered autoloaders and iterate over them…

Detect Composer ▶

The Magic™

class AutoloaderOverloader

{

// ...

public static function init(string $filterName): void

{

$loaders = spl_autoload_functions();

foreach ($loaders as &$loader) {

// Unregister each loader, so they can be re-registered in the same order.

spl_autoload_unregister($loader);

if (is_array($loader) && $loader[0] instanceof ComposerClassLoader) {

// Replace Composer autoloader with our hijacking autoloader.

$loader[0] = new self($loader[0], $filterName);

}

}

// Processed all the loaders, re-register them with PHP in their original order.

foreach ($loaders as $loader) {

spl_autoload_register($loader);

}

}

}

If we detect that one of the autoloaders is Composer…

Decorate and replace ▶

The Magic™

class AutoloaderOverloader

{

// ...

public static function init(string $filterName): void

{

$loaders = spl_autoload_functions();

foreach ($loaders as &$loader) {

// Unregister each loader, so they can be re-registered in the same order.

spl_autoload_unregister($loader);

if (is_array($loader) && $loader[0] instanceof ComposerClassLoader) {

// Replace Composer autoloader with our hijacking autoloader.

$loader[0] = new self($loader[0], $filterName);

}

}

// Processed all the loaders, re-register them with PHP in their original order.

foreach ($loaders as $loader) {

spl_autoload_register($loader);

}

}

}

We decorate it, replace it…

Re-register ▶

The Magic™

class AutoloaderOverloader

{

// ...

public static function init(string $filterName): void

{

$loaders = spl_autoload_functions();

foreach ($loaders as &$loader) {

// Unregister each loader, so they can be re-registered in the same order.

spl_autoload_unregister($loader);

if (is_array($loader) && $loader[0] instanceof ComposerClassLoader) {

// Replace Composer autoloader with our hijacking autoloader.

$loader[0] = new self($loader[0], $filterName);

}

}

// Processed all the loaders, re-register them with PHP in their original order.

foreach ($loaders as $loader) {

spl_autoload_register($loader);

}

}

}

Before re-registering each autoloader in their original order.

Usage (index.php) ▶

The Magic™

// index.php

require_once __DIR__ . '/vendor/autoload.php';

stream_filter_register('my_compiler_filter', MyCompilerFilter::class);

AutoloaderOverloader::init('my_compiler_filter');

To make all of this come to life, we…

Load Composer ▶

The Magic™

// index.php

require_once __DIR__ . '/vendor/autoload.php';

stream_filter_register('my_compiler_filter', MyCompilerFilter::class);

AutoloaderOverloader::init('my_compiler_filter');

Load Composer…

Register stream filter ▶

The Magic™

// index.php

require_once __DIR__ . '/vendor/autoload.php';

stream_filter_register('my_compiler_filter', MyCompilerFilter::class);

AutoloaderOverloader::init('my_compiler_filter');

Register our stream filter, and…

Initialize autoloader ▶

The Magic™

// index.php

require_once __DIR__ . '/vendor/autoload.php';

stream_filter_register('my_compiler_filter', MyCompilerFilter::class);

AutoloaderOverloader::init('my_compiler_filter');

Initialize our autoloader overloader. That was a lot of code to throw at you, but that’s it for our overloader!

Carry on as normal ▶

The Magic™

// index.php

require_once __DIR__ . '/vendor/autoload.php';

stream_filter_register('my_compiler_filter', MyCompilerFilter::class);

AutoloaderOverloader::init('my_compiler_filter');

$class = new MyClass;

Now we carry on developing as normal - any class loaded after initializing our autoloader will have it’s source code passed through our stream filter!

Other projects using autoloading ▶

$executableCode = transformCode($sourceCodeFile);

There are other projects out there that implement autoloader overloading

eval()

$executableCode = transformCode($sourceCodeFile);

eval($executableCode);

Some evaluate the generated code

Function usually disabled ▶

$executableCode = transformCode($sourceCodeFile);

eval($executableCode);

eval() is evil* and is disabled on hardened PHP installations.



* is what we’re doing here any better though? 🙄

But evaluating strings as code using the eval() function is usually disabled, or at least highly discouraged.

Because what we’re doing is sooo much better… 🙄

Write to disk ▶

$executableCode = transformCode($sourceCodeFile);

eval($executableCode);

file_put_contents(

$compiledFile,

$executableCode

);

require $compiledFile;

eval() is evil* and is disabled on hardened PHP installations.



* is what we’re doing here any better though? 🙄

Others write the generated code to disk before requiring the new file.

Won’t work on read-only ▶

$executableCode = transformCode($sourceCodeFile);

eval($executableCode);

file_put_contents(

$compiledFile,

$executableCode

);

require $compiledFile;

eval() is evil* and is disabled on hardened PHP installations.



* is what we’re doing here any better though? 🙄

But this only works if PHP has permissions to write to disk.

But this won’t work on read-only filesystems, such as sandboxes containers where cache is already warmed up on deployment.

Streams most elegant ▶

STREAMS

AWESOME

ARE

Using stream filters is perhaps the most elegant. It’s built into PHP’s core functionality and just works.

If already into meta-programming by manipulating source code, streams aren’t a challenge.

Now we know, what next? ▶

What Next?

Now that we know we can modify the PHP code that actually gets executed by PHP on-the-fly from within the application we are running, what’s next? What important work can we do given all this power?

Short closures! ▶

Short Closures!

PHP 7.4 … y so slow?

That’s right! Implementing short closures because we’re too impatient to wait for 7.4!
We’ve learnt how it can be done, so let’s roll our own!

Pre ▶

Pre

Christopher Pitt’s Preprocessor library takes invalid PHP code that looks like:

And turns into ▶

Pre

$oddLetterCountWords = array_filter($fruits,
return strlen($fruit) % 2;
);

($fruit) => {

}

This, and turns it into valid PHP code that looks like:

This. Pretty unremarkable ▶

Pre

$oddLetterCountWords = array_filter($fruits,
return strlen($fruit) % 2;
);





$oddLetterCountWords = array_filter($fruits,
return strlen($fruit) % 2;
);

($fruit) => {

}





function ($fruit) {

}

This. It’s a pretty unremarkable change, but we’re transpiling invalid code into something that PHP will understand.

Building on the knowledge we’ve already gone over, implementing this is a two-step process:

Step one ▶

$ composer require pre/short-closures:^0.8

Step one: include the library as a dependency.

Step two ▶

$ composer require pre/short-closures:^0.8



// Alias for implementing and registering a stream filter.
public function transformCodeFilter($sourceFile) {
return \Pre\Plugin\parse(file_get_contents($sourceFile));
}

Step two: use it inside your stream filter (remembering that this logic is an alias for implementing and registering a stream filter).
And we’re done!

Established any input ▶

Obfuscation

We’ve established that as long as we return a string containing valid PHP code, we can accept pretty much any input to manipulate even if that input is nothing.

Such as encrypted ▶

<?php exit('Source Code Protected.'); ?>

aFcZG1BJF0UNAxVSBAgAEREcERF/HxwJRRJTVQlSeSoADhlFHhUADREANR9QLyYMGwYXAxVMABxYeHMFB0VUOxxNFQAbCz8mTw4fCwtLOhw1O1QdA2YBGhpEAFQaCg0pIABTGwoXUwRVbioKH0EdHFRkCAMAGxhUNwBOBxcMGR4AHnNbb05DUlkAAUIYAQYAFRobERcMTw1POzpHOh4GGEVBIUUfGhFTFQBXFwYEBwBTH0xDADMLF1AGHVMLTw8qTUVBTlQAVE9SFhEWBxxFAhxXRTwGAQkfGlMRQEJoFgMZHU9FdwwdCAEPc0FSeQBJUwATZQk=

Such as input like encrypted source code!

Which can be decrypted on-the-fly… ▶

<?php declare(strict_types=1);

namespace App\Controller;

use Symfony\Component\HttpFoundation\Request;

use Symfony\Component\HttpFoundation\Response;

class DefaultController

{

public function __invoke(Request $request): Response {

return new Response('Hello, World!');

}

}

Which can be unencrypted on-the-fly into something resembling a normal PHP file.

Stream filter method ▶

function transformCodeFilter(string $sourceFile): string {

$source = file_get_contents($sourceFile);

return strpos($source, PREAMBLE_TEXT) !== 0

? $source

: xorStringWithKey(

base64_decode(substr($source, strlen(PREAMBLE_TEXT))),

ENCRYPTION_KEY

);

}

function xorStringWithKey(string $source, string $key): string {

$output = '';

for ($i = 0; $i < strlen($source); $i++) {

$output .= $source[$i] ^ $key[$i % strlen($key)];

}

return $output;

}

Once again, we have a fairly simple-ish stream filter method to what once seemed like something that could only happen using extensions.

Grab source code ▶

function transformCodeFilter(string $sourceFile): string {

$source = file_get_contents($sourceFile);

return strpos($source, PREAMBLE_TEXT) !== 0

? $source

: xorStringWithKey(

base64_decode(substr($source, strlen(PREAMBLE_TEXT))),

ENCRYPTION_KEY

);

}

function xorStringWithKey(string $source, string $key): string {

$output = '';

for ($i = 0; $i < strlen($source); $i++) {

$output .= $source[$i] ^ $key[$i % strlen($key)];

}

return $output;

}

We grab the source code just like every other transformation we’ve done.

Return untouched ▶

function transformCodeFilter(string $sourceFile): string {

$source = file_get_contents($sourceFile);

return strpos($source, PREAMBLE_TEXT) !== 0

? $source

: xorStringWithKey(

base64_decode(substr($source, strlen(PREAMBLE_TEXT))),

ENCRYPTION_KEY

);

}

function xorStringWithKey(string $source, string $key): string {

$output = '';

for ($i = 0; $i < strlen($source); $i++) {

$output .= $source[$i] ^ $key[$i % strlen($key)];

}

return $output;

}

If we don’t need to decrypt the source code, we immediately return the it untouched.

Pass to decrypt function ▶

function transformCodeFilter(string $sourceFile): string {

$source = file_get_contents($sourceFile);

return strpos($source, PREAMBLE_TEXT) !== 0

? $source

: xorStringWithKey(

base64_decode(substr($source, strlen(PREAMBLE_TEXT))),

ENCRYPTION_KEY

);

}

function xorStringWithKey(string $source, string $key): string {

$output = '';

for ($i = 0; $i < strlen($source); $i++) {

$output .= $source[$i] ^ $key[$i % strlen($key)];

}

return $output;

}

But if we do detect the source code needs decrypting, by looking for the exit statement at the beginning of the file (called the preamble) we pass it to another function to decrypt it.

Iterates over each byte ▶

function transformCodeFilter(string $sourceFile): string {

$source = file_get_contents($sourceFile);

return strpos($source, PREAMBLE_TEXT) !== 0

? $source

: xorStringWithKey(

base64_decode(substr($source, strlen(PREAMBLE_TEXT))),

ENCRYPTION_KEY

);

}

function xorStringWithKey(string $source, string $key): string {

$output = '';

for ($i = 0; $i < strlen($source); $i++) {

$output .= $source[$i] ^ $key[$i % strlen($key)];

}

return $output;

}

Which iterates each byte and xor’s it with the corresponding byte in an encryption key.

This isn’t AES - it’s meant to be simple. Also happens to be how GitHub protects their Enterprise source code.
People always going to reverse engineer something if they really want, XOR encryption should be enough prove intent in court of law.

Performance: remember still just experiments ▶

Performance like it’s 1994 🎉

But remember that this is an experiment. Caching the transformation result from decrypting source code would be completely pointless.
Even with the Just-In-Time compiler coming in PHP8, source transformation or manipulating ASTs on the fly is never going to be fast.

But that’s not the point ▶

But that’s not the 📌

But that’s not the point.

You can do some amazing things in PHP that is was never designed to do.

Pre transpiles ▶

But that’s not the 📌

  • Pre transpiles new language features just like BabelJS

Christopher Pitt’s Pre library transpiles new and non-existant language features just like BabelJS

AOP injects logic ▶

But that’s not the 📌

  • Pre transpiles new language features just like BabelJS
  • Go! AOP allows you to inject logic on-the-fly

AOP allows you to inject logic at points in your code on-the-fly.

Source transformation manipulate existing code ▶

But that’s not the 📌

  • Pre transpiles new language features just like BabelJS
  • Go! AOP allows you to inject logic on-the-fly
  • Source transformation means you can execute non-existent code

Using source transformation and autoloader overloading you can not just manipulate existing code on-the-fly, but generate entirely new classes with code templates.

ReactPHP asynchronous PHP ▶

But that’s not the 📌

  • Pre transpiles new language features just like BabelJS
  • Go! AOP allows you to inject logic on-the-fly
  • Source transformation means you can execute non-existent code
  • ReactPHP uses stream_select() asynchronously

ReactPHP allows you to write asynchronous code by using the native stream_select() function

Icicle generators ▶

But that’s not the 📌

  • Pre transpiles new language features just like BabelJS
  • Go! AOP allows you to inject logic on-the-fly
  • Source transformation means you can execute non-existent code
  • ReactPHP uses stream_select() asynchronously
    • Icicle uses generators

And Icicle does it using generators.

There are so many projects out there that push PHP beyond the limits of what people thought was possible for the language and the likelihood that the authors of these projects specifically knew what they were going to build ahead of time is very low.

Question. ▶

Question.

Question what’s possible

Experiment ▶

Question.

Experiment.

Experiment with ideas

Have fun. ▶

Question.

Experiment.

Have fun.

Have fun, like you did when you first became a developer.

You won’t know what’s possible until you push past the limits of the language.

I want you to leave this room thinking that this talk didn’t go into enough detail, and that you’re curious to know more.

Overhearing conversations next year ▶

hey guess what I got working!

DPC 2020

I look forward to overhearing people’s conversations at DPC 2020 after a year of experimenting saying “guess what I got working!”

Thanks! ▶

Thanks!

@ZanBaldwin
Intergalactic Agency, Vancouver

joind.in/talk/c9f62

Slides are on joind.in!

Fin.

Say THANK YOU!
Are there QUESTIONS?

Presentation - Google Slides