Pixly: User Manual
Pixly is a application built with the Blockly block programming library. Introducing blocks specific to canvas image manipulation, Pixly allows people to create syntactically correct programs with blocks to read and change the pixels of images. This allows for all sorts of media computation applications like red-eye removal, green-screening, negatives, and more.
I. Layout of the Pixly Application
When Pixly begins, you will see a large image in the Display Canvas on the left of the screen, with some buttons and smaller images below, and the Scripting Area on the right of the screen.
The scripting area on the right is where you will drag and drop your blocks to create your pixly program. The display canvas on the left is where you will see the result of your image manipulation program.
II. Selecting, Adding and Removing Canvases
Pixly has multiple canvases displayed in the Canvas Selection Area. You can click on any one of these to make that canvas displayed as the display canvas. Each canvas in the canvas selection area is numbered, and this number is the one you will reference in your blocks to specify which canvas(es) you are programmatically changing.
Blank canvases can be added with the +canvas button in the Canvas Buttons area, and the currently selected canvas can be hidden with the -canvas button.
III. Uploading User Images to the Canvas
It is possible to upload images from your own hard drive to edit.
There is a button labeled “Upload Image”. Clicking on this button will open up a file explorer dialogue that will allow you to select the image you wish to upload onto your Pixly Canvas.
IV. Exporting/Importing Projects and Saving the Canvas
Immediately above the scripting area are the Project Management buttons. One is labeled Import Project, and one is labeled Export Project. The Import Project button will let you load several example programs available by default in Pixly, or upload a json file containing a project that you or someone else saved from Pixly.
The Export Project button allows you to save your Pixly project (which includes the blocks creating your code in the scripting area, along with any images you have uploaded into Pixly from your computer) as a json file. You can share this json file with others (by email?), and they may Import it, allowing them to see your full Pixly project!
In addition to Exporting your entire project, the button with a camera on it immediately under the Reset Canvas button will allow you to save the currently selected canvas as an image onto your computer.
V. Pixly Specific Blocks
Now we’ll go through all of the blocks custom created for the Pixly media computation application.
(A Note for Users of the Previous Version of Pixly: Previously, there were multiple blocks stuck together into pre-existing loops that would iterate through the entire length of the canvas, using the width and height of the canvas to get each pixel. These have been removed from this version of Pixly, due in part to the limitations of relying on hard coded loops, and also due to the availability of those loops in the “Loops” block category. Later on this manual there will be a mini-tutorial going over how to construct one of the program examples that will go through the process of recreating this step by step).
A. Blocks under the Canvas tab
The first block under the Canvas tab is the Canvas block. This block allows you to select a canvas from the dropdown menu (indicated by a downward arrow, in the bottom right of the block). Depending on which Canvas you select, a miniature display of that canvas will be shown. This block can be used wherever a Pixly block indicates an empty space for a “canvas”.
(Note: although, you can use this canvas block for any “canvas” slot, it is also acceptable to simply use a number block that indicates the number of the canvas you wish to reference.)
Similar to the block above, the selected canvas block can be used wherever a reference to a canvas in another block is necessary. However, instead of referring to a specific, dedicated canvas, this block will return the number of whatever canvas the user is currently looking at during program execution.)
The width/height block. Whenever you are using Pixly and operating on a canvas (for instance: constructing a loop on the pixels on a canvas), it might be helpful to know the width or height of that canvas without having to measure it yourself. This block will return either the width or the height of the canvas depending on which option is selected from the dropdown menu. The input on the right of the block should either be a number referring to the canvas you wish to get the width/height of, or one of the canvas blocks above.
The reset canvas block. This block simply redraws the original image of the canvas. For instance, if you edit the green dog image on canvas 1, but wish to have the original image back again, you can use this block. The input on the right of the block should either be a number referring to the canvas you wish to get the width/height of, or one of the canvas blocks above.
The pixels from canvas block. This block will return a list of all the pixels in the specified canvas. This list is a single-dimensional list containing pixel objects (which will be covered in more detail under the “Pixels” blocks section). For instance, for an image that is 100 x 100 pixels, the 1st item of the list will be the topmost, leftmost pixel. The 100th item of the list will be the topmost, rightmost pixel. The 101st item of the list will be the pixel that is one below the topmost, leftmost pixel (coordinates of x: 1, y: 2). Et cetera.
B. Blocks Under the Pixels Tab
The get pixel block. This block will return a Pixel object from the specified x and y coordinates from the specified canvas. The x and y inputs can be number blocks from the Math tab to specify a specific x and/or y coordinate, or they can be filled with variables (this will be shown in the examples below).
A pixel object is a representation of a pixel on a canvas image, and is composed of three values: r, g, and b, which correspond to the red, green, and blue values of the specific pixel. (http://en.wikipedia.org/wiki/RGB_color_model for more information). These values can be retrieved from other blocks in the pixel tab.
This block will change the pixel at the specified x, y coordinate on the specified canvas to specifed value.
The pixel value can either be a pixel variable (for instance, pixel), or a colour block from the colour tab). This block can be used in conjunction with the main media computation loop to fully or partially copy one canvas onto another.
This block will change the specified pixel to the specified color value. The pixel on the left needs to be a pixel variable created from the get pixel block (either directly, by inserting the get pixel block into the left slot, or indirectly, by first setting a variable to the output of the get pixel block. The value on the right can either be a color block from the Colour tab, or another pixel (in which case the pixel from the left input will change its color to the color of the pixel on the right input).
This block will return the colour value of the specified pixel. The color value acts the same as any colour block from Blockly’s colour tab, and is represented in code as a hexadecimal string of the color.
This block will return the red, green, or blue value of a pixel. RGB values range from 0 to 100.
This block will return the red, green, or blue intensity of a pixel. Color intensity is calculated as follows:
R Intensity: (red value) / ((blue value + green value) / 2)
G Intensity: (green value) / ((blue value + red value) / 2)
B Intensity: (blue value) / ((red value + green value) / 2)
This block is very handy for red-eye removal, green-screening, or other algorithms that rely on relative color intensities.
This block can be used to set the red, green, or blue value of a pixel. RGB values range from 0 to 100.
For instance, if you wanted to make a picture be tinted cerulean (get rid of all the red in the picture), you would use this block to set the red value of every pixel to 0, which would still maintain their blue and green values.
C. Blocks under the Events tab
The first block under the media comp tab, and the only block that is placed on the scripting area whenever Pixly is started. This block is where any blocks that you want to be executed when you press the run button need to be placed.
Any blocks that are outside of this run button block are considered scratch and will not be have any direct effect on the canvases.
(NOTE: an exception is any function created outside of the run block. These will not be executed directly, but can be referenced from inside of the run block and will function properly).
The console log block may not be helpful for everyday usage of Pixly. It does not operate at all on the canvases or pixels, but may be helpful for when you are debugging your program. Whatever value is placed in the input to this block will be printed to the JavaScript console of your web browser (see here for instructions on accessing the JavaScript console). This may be helpful for seeing what the value of a variable is at a given point in the program.
(NOTE: Be careful about using this block inside a loop iterating through all the pixels of a canvas. This will greatly slow down the execution of the loop, as your web browser takes some time to print out values to the console).
VI. Code Examples (Tutorial)
(NOTE: This section previously included several screenshots of example code, with accompanying before/after pictures of the images. Instead, this section will now go over a small example demonstrating some helpful Pixly basics. The more advanced examples are included in the Import Project dialog, under “Choose an Example Project” dropdown menu.)
A. Chromakey (Cerulean): In this example, we will go over the steps to create a program that will remove all the red value in an image. It will do this by iterating through every pixel in an image, and changing the red value of each of these pixels to 0. The resulting image will look like this:
1. When you first start up Pixly, you should see a When Run Program block in the Scripting Area
If you do not for whatever reason, go ahead and drag that block from the Events block category onto your Scripting area.
2. Next, we need a way to go through every pixel of the canvas that we will choose. For this, the simplest way is to use a loop! First off, drag a for each item in list loop from the Loops category:
Now, while this loop is very useful to iterate through every pixel in the canvas, it currently is not set up to do so. In order to set it up, we need to specify the list that we are looping through. We will make this list be the list returned from the get pixels from canvas block from the Canvas category. Additionally, you will have to specify the canvas that you are getting the pixels from. We will do this with a canvas block (also from the Canvas category).
3. Now we have our program set up to loop through all the pixels from a canvas. However, this is still not enough. We need to still do some manipulation of each of these pixels. In this example, we want to change the red value of every pixel to zero. We can accomplish this by using the change red value of pixel block under the Pixels category! Finally, we need to make sure that the variable name in the loop is the same as the variable name in the change red value of pixel block. To do that, click the dropdown menu on the for each item i to change the name of the variable i to pixel. This ensures that the variable being assigned to each pixel in the canvas is named the same as the pixel variable we are trying to change the red value of.
4. If you click the Run Program button at this time, you’ll see the blocks in your scripting area start to highlight over and over again in the for loop. If you pay close attention to Canvas 0 on the left (the Display Canvas), you should see the pixels at the top of the image being changed to a light blueish color. However, this is happening very slowly. That is because Pixly is set up to highlight every block while it is operating, in order to show the user the sequence of events that their program is orchestrating. While this may be helpful for some scenarios, it is definitely not the most helpful when you are waiting such a long time for the image to be fully changed by your program.
In order to disable this feature for loops, I have included a “run max speed” checkbox option on the bottom of each loop. Go ahead and click that option in your program, and then click Run Program. If you’ve followed along successfully so far, your program should run and change the baby to a light blueish tint, as shown below:
5. Bonus: However, in some applications, it might be helpful to keep some loops unchecked, so that they run at a normal speed. For instance, modifying the program we’ve just created to use two for loop instead of directly iterate through every pixel of a canvas, iterate through the pixels of a canvas by specifying their x and y coordinates like as follows:
The result of running that program with only the inside loop running at max speed will look like the follows:
