Workflow Refining

Managing projects in Pixinsight is one of the most powerful tools of this program, but unfortunatelly there is not much literature about it. This is the motivation to write this tutorial, in which we will see the benefits and the procedures involved. It is widely known that using projects allows to store the processing workflow in the History Explorer, but there’s also the possibility of refining the entire workflow.

1.What’s a Project?

Projects in PixInsight can be considered as a “snapshot” of the current state of the program.

Let’s assume the following state, where we have started by loading some images and working on them, generating some masks, process icons and even active instances of processes.

If we go to File->Save Project… an emergent window will be displayed and we will be able to choose some characteristics that we will left as is for now. We will only assign a name and a path for our project and save it.

We will see as the Process Console writes lines indicating the elements that are being saved. Note as it saves workspaces, images, process icons…

Now we can close PixInsight, open it again and use Load Project… to load our saved project.

Depending on the size of the images (mainly) we will have to wait some seconds, or minutes. Once loaded we will se as all the elements have been restored, but not only that, but they are located in their corresponding workspace, and in their corresponding location inside the workspace. Everything is exactly where we left it. Cool right? Well, read on and you will discover many more useful things that can be done with projects.

As stated before, when we save a project we are saving mainly process icons and images, but there are other functionalities related with the use of projects. Let see each of this features while we explain details about how they work and what we can do with them.

2.Process Icons

One of the PixInsight differences against other processing software (like Photoshop) is that you can have as many process instances actives as you want and you don’t have to close them in order to continue interacting with other elements. There’s a limitation here: only one instance per process can be active, so if we load (by double clicking on the process icon, for example) another instance, it will modify the current active instance of the process instead opening a new active instance. Let see things that we can do with process icons.

1.1.Processing Summary

For our purposes about managing a project, process icons can be useful to have a summary of the processes used in a workflow. We will see in a moment that there are other ways, maybe better ones, to track our workflow.

As seen in the image, processes icons instances have been used to create a list in a visual way where we can rename them, and group with the corresponding masks. Note also how a special process called “No Operation” has been used as a heading to group the different processing steps.

1.2.Save/Load Process Icons

Process Icons can be saved, as individuals or grouped, and can be loaded to be used in different projects.

To save Process Icons you can select them and right click on one of them. Then, in the emergent window, select Save Selected Icons…

They will be stored as a *.xpsm file. To load them again, right click in an empty space of the workspace and select Load Process Icons or Merge Process Icons. Load will load them and discard the existing processes on the whole project, while Merge will add this new processes.

Notes:

-The names of the loaded process icons will be renamed if there are existing process using the same name tag.

-Loaded process Icons will be placed in the same workspace that they were created. If the workspace doesn’t exist in the current project, it will be created by Pixinsight.

-Be carefull to choose Merge instead of Load if you want to add additionl processes to your project. An accidentally load (and therefore discard of the current Process Icons) can’t be undone.

1.3.Updating Process Icons

Sometimes we may want to modify the parameters of a Process Icon instance, for example in a refining procedure were we have fine tuned the parameters to achieve a better result. To update a process Icon we must drag the improved process over the process Icon.

The program will ask for confirmation to overwrite the parameters of the target Process Icon because it can’t be undone.

3.History Explorer

We have seen how the Process Icons can be used as a quick way to store specific parameters of the tools provided by PixInsight, and how can they be useful to track our processing workflow, but there’s a functionality, linked to the use of projects, that will give us an improved view of the workflow used on a specific image.

Meanwhile an image is embedded on a project, we will be able to see the historic of all the processes applied to the image by using the History Explorer. The History Explorer is normally located in one of the left side tabs. If it is not present, or if we accidentally closed it, we can activate it going to the top menu Edit, and selecting them on the Processes submenu. Once activated we can visualize it by clicking in the corresponding tab.

3.1.Overview

The History Explorer window has a main view where we will found every process applied to the selected image. To select an image to be seen in the History Explorer we can use the bottom image selector and search the image, although the quickest procedure (specially in a project with many images involved) is to right click in the image (it must be active) and select History Explorer.

Once the image is selected, the processes will populate the main window. They are sorted as they have been applied so on top we will see the first process applied.

Going in detail, the first line shows the Root (<Initial State> in the example), indicating that is the initial state of the image. Going down, the processes appear in the order they have been applied. Every process has a number on the left, indicating the position on the list. On the most left we can see an arrow, that indicates the current state that the image is showing, so if we undo a step, the arrow will go up one position on the list. A quick undo/redo action can be performed by double-clicking the desired position (it is important to double-click on the column where the arrow is placed). This functionality allows us to go back and forth several steps on the workflow very quicky in order to analyze the results.

This is the basic functionality of the History Explorer: to act as an advanced undo/redo feature, with visual feedback, but it can offer us some other interesting features.

3.2.Sticking the History Explorer

All tab windows can be sticked by clicking in the small square on the top right corner of the window. It seems a silly advice, but it is very useful when performing a workflow refining procedure (refining procedure will be covered at the end of this article).

3.3.Copy/Tranfer Process

Processes contained in the History Explorer can be reused to be applied in other images or to simply keep them as a Process Icon in our workspace.

To do it we have 3 alternative procedures:

-Drag a process from the History Explorer to the workspace. This will create a new Process Icon instance that can be saved, reused…

-Drag a process from the History Explorer to an image. This will apply the process, with the stored parameters (but not the mask involved) to the target image.

-Double click on the process to open a process instance with the stored parameters, to further applying it on other image, or to tweak the parameters before applying it.

3.4.Code Visualization

Using the Show Extension button we can visualize the internal code of the selected process (not the whole workflow). It can’t be modified from the History Explorer, so it is only an informative panel and needs some kind of knowledge about programming to understand it. It can be useful to check the parameters of a certain process application, but I didn’t found much more utility than that. In any case, it is worth to mention because as we will see, a Process Container derived from a History Explorer effectively can be modified.

3.5. History Explorer instances (Process Container)

The History Explorer includes the same button used to drag a process into an image. It is very important because allows to make a copy of the whole History Explorer by creating a Process Container that contains, not only the information of the processes and its order, but also the involved masks.

This is great and we will see later how to get the most from this feature, but I advance that, once the Process Container instance is created, we can apply it to another image (a copy of the original, not processed image, for example) and it will apply all the active processes with the corresponding masking to it.

3.6.History Explorer on duplicated images

Something we may want to make a copy of an image in a intermediate state of the workflow, for example to try an alternative processing. Let’s assume that we have done this, and we have applied a better alternative processing that we want to keep. If we open the History Explorer of this duplicated image we will notice that only the recently added processes are visible on the list, but we may notice that there’s a black arrow on the left side of the first row, called in this case <Initial State>.

We can click on it and we will see the processes applied before the duplication of the image. This steps are embedded in the Initial State and cannot be undone, but we can drag them to the workspace in order to reuse them.

4.Process Container

We have seen how a History Explorer can be entirely duplicated and how PixInsight creates it as a Process Container. Process Container is a very powerful tool that can be used, not only to analize a determined processing workflow, but also to execute it into an image to apply several processes, with their corresponding masks, in one step. In addition, the Process Container’s content can be modified and this is a wordeful way to try different processing alternatives. Let see how it works and which possibilities do it offers.

4.1.Overview

As for the History Explorer, we will start by analizing the tool to see which functionalities it has.

The main window of the Process Container tool reminds to the History Explorer as it is composed by a table, populated with processes, in a certain order, and with their masks linked.

The most left column in this case indicates which processes are active or disabled. This state can be toggled by the user. In the bottom area of the list there are a series of buttons that allows us to enable/disable, rearrange, as well as to delete the selected process.

Note: There’s no possibility (as far as I know) to select more that one process at a time, so each of this actions should be performed one by one.

There are also buttons to drag a new instance, to be applied on an image or directly on the workspace to create a new Process Icon instance.

4.2.Direct application of a Process Container

A Process Container derived from a History Explorer can be applied to an original image to achieve an equivalent result. To do it is as easy as to drag the process to the image.

If we compare the original processing with the one applied throught Process Container we will notice that the processes have been grouped inside a one step process called (obviously) ProcessContainer. It is important to note also that if we undo/redo, we will be only able to redo/undo the whole group.

4.3.EDITING THE SOURCE CODE

There is an “extension” button on the bottom right that will show us the code, as in the History Explorer, with one major difference: in this case the code can be modified by using the Edit Code Button. This is an advanced, but very powerful feature that may delight to those used to working with codes. If we click on it, a new window will appear showing the code on the entire Process Container (taht means, all the processes involved on it).

5.EXAMPLE OF A REFINEMENT WORKFLOW

Let’s assume that we have finished the processing of our image. We have applied a serie of processes, trying to improve certain aspects of the image and we have a final result. The processes are applied one on top of the rest and sometimes the last processes bring up defects that may be hidden before. This fact brings the necessity to go back on the processing. We could start from scratch, taking again the unprocessed image and applying the workflow processes one by one, but for workflows of 30-40 processes, the task can be quite boring, specially if there is the necessity of manking an iterative refinement. Fortunatelly we can use the procedures described in this article to make this actions in a simple and quick way.

We will use the following image as our starting point:

It is an image on which we applied three basic processes: star erosion (masked), saturation, and stretch. We clearly went too far with the saturation, but let’s assume we didn’t realize because we applied it before the stretch (…and we weren’t aware of what STF is…). In that case we may want to refine this workflow. Saturation is obviously the first process to refine, but we want to see how the stretch affects it. To do that, we can proceed as follows:

Open the corresponding History Explorer and create an instance (that is a Process Container, as explained before).

Now, we need the original “unprocessed” image (unprocessed here means the image before the workflow we are studying. This is important to be noted, as this image could be processed, for example in the linear stage, and we would only want to improve our nonlinear processing). If we haven’t kept an image copy of this state, we can undo our study image to its <Initial State> and then make a copy.

Keep the unprocessed image as a master and apply the improvements over duplicates of it. We will name it as LRGB_V0.

With all this elements, I propose to prepare the workflow as in the following image.

So we have our processed image (LRGB_V1) in the center. Its corresponding History Explorer (Left Bottom), opened and sticked. A Process Container icon of the LRGB_V1 processing, and an active instance of this Process Container too.

5.1.WORKFLOW REFINEMENT: UNMASKED PROCESSES

To make a new workflow refinement first we will clone the original unprocessed image (LRGB_V0) to work on it. Then double clic on the Color Saturation process on the Process Container. An active instance will be opened.

The saturation value is 2.327. We will try with a value of 1. To do this, change the parameter in the active instance and drag the process to the process container. It is useful to first disable the process we want to substitute in the process container becasue as the processes inside the process container loss their identifier it can be confusing to have two identical processes one next to the other.

Now we can drag the improved Process Container over the unprocessed clone image and see the result. We will rename the result as LRGB_V2. In the following image the side-by-side comparison shows a significant improvement, with much more pleasant colors.

The History Explorer shows the LRGB_V2 image and it shows as the workflow has been applied as a one-step process, with subprocesses (expressed with a subindex 1.1, 1.2,…) inside it. It is important to remind that this processes can’t be undone one by one. If we aren’t satisfied with the result we can undo one step to return to the unprocessed version. This procedure can should be donde iteratively until we find the correct value. Once satisfied, we can delete the old process with the correesponding button. The key Supr doesn’t work here.

5.2.WORKFLOW REFINEMENT: MASKED PROCESSES

We have seen how to refine a simple application of a process (that means, unmasked) inside a workflow, but to refine masked processes is a little bit more complicated. The problem comes from the fact that process active instances or process icons doesn’t had the information of being masked by itself, as is the image which provides this information to the processing. But to transfer a process and its corresponding mask to a process container is possible by using the History Explorer. Processes dragged from a History Explorer to a Process Container will effectively transfer the masking information. We can use this property to create a dummy image that we can mask and apply refined processes (or use refined masks with existing processes). This dummy image should have (just for precaution) the same properties of the working image (size, channels, bits…) but it can be simply black, so we will simply create a new image and clic on Set as Active Image (be careful to previously activate the target image).

The following image shows the layout for this procedure.

So we start with 4 images: the original image, unprocessed, the processed version using the original mask, the original mask itself, and an improved mask that we want to substitute the original mask. We will double clic on the desired process (Morphological Transformation in this example) to have an active view of it and we will do:

  1. Mask the Dummy Image with our improved mask. Be sure to invert it if necessary.
  2. Apply the desired process to the Dummy Image.
  3. A new process will appear on the History Explorer of the Dummy Image. The process should appear masked with the improved mask.
  4. Drag the new process from the History Explorer to the Process Container.

We will then disable the previous erosion process and rearrange the new one. Once done, we can apply the new Process Container (our improved workflow) to a clone of our unprocessed image and check the result:

The improved mask has clearly produced a better star shape after the erosion process.

It is important to note that this procedure can be used to replace a mask, and also to replace a process by an improved one, reusing the corresponding mask.

5.3.ANNEX: REFINEMENT OF MASKED PROCESSES USING THE SOURCE CODE EDITOR

In case of only replacing a mask of a given process, we can use the Source Code Editor to quickly make the change without the need of a Dummy Image. In the following image we can see a Process Container (our original workflow) using the mask ero_msk_v0. If we clic on the Edit Instance Source Code button, a new window will open showing the Source Code of the Process Container. Below the Morphological Transformation fuction, we can see the instruction P001.setMask(0,»ero_msk_v0) indicating the mask usage.

We can simply rewrite the mask name to the desired mask and press the apply button on the Edit Source Code header.

5.4.RESULT

This is the result of the refinement process described in this article plus an additional reduction of brightness has been applied to the last Histogram Transformation.

Left: Before Refining Workflow
Right: After Refining Workflow

Deja una respuesta

Introduce tus datos o haz clic en un icono para iniciar sesión:

Logo de WordPress.com

Estás comentando usando tu cuenta de WordPress.com. Salir /  Cambiar )

Imagen de Twitter

Estás comentando usando tu cuenta de Twitter. Salir /  Cambiar )

Foto de Facebook

Estás comentando usando tu cuenta de Facebook. Salir /  Cambiar )

Conectando a %s