pymepix.processing.logic package

Submodules

pymepix.processing.logic.centroid_calculator module

class pymepix.processing.logic.centroid_calculator.CentroidCalculator(cent_timewalk_lut=None, number_of_processes=4, clustering_args={}, dbscan_clustering=True, *args, **kwargs)

Bases: pymepix.processing.logic.processing_step.ProcessingStep

Class responsible for calculating centroids in timepix data. This includes the calculation of the clusters first and the centroids. The data processed is not the direct raw data but the data that has been processed by the PacketProcessor before (x, y, tof, tot).

process(data):

Process data and return the result. To use this class only this method should be used! Use the other methods only for testing or if you are sure about what you are doing

calculate_centroids_cluster_stream(chunk)

calculate_centroids_dbscan(chunk)

calculate_centroids_properties(shot, x, y, tof, tot, labels)

Calculates the properties of the centroids from labeled data points.

ATTENTION! The order of the points can have an impact on the result due to errors in the floating point arithmetics.

Very simple example: arr = np.random.random(100) arr.sum() - np.sort(arr).sum() This example shows that there is a very small difference between the two sums. The inaccuracy of floating point arithmetics can depend on the order of the values. Strongly simplified (3.2 + 3.4) + 2.7 and 3.2 + (3.4 + 2.7) can be unequal for floating point numbers.

Therefore there is no guarantee for strictly equal results. Even after sorting. The error we observed can be about 10^-22 nano seconds.

Currently this is issue exists only for the TOF-column as the other columns are integer-based values.

centroid_chunks_to_centroids(chunks)

centroids = [[] for i in range(7)] for chunk in list(chunks):

if chunk != None:

for index, coordinate in enumerate(chunk):

centroids[index].append(coordinate)

cluster_stream_preprocess(shot, x, y, tof, tot)

cs_max_dist_tof

Setting the maximal ToF distance between the voxels belonging to the cluster in Cluster Streaming algorithm

cs_min_cluster_size

Setting the minimal cluster size in Cluster Streaming algorithm

cs_sensor_size

Setting for the number of packets skipped during processing. Every packet_skip packet is processed. This means for a value of 1 every packet is processed. For 2 only every 2nd packet is processed.

cs_tot_offset

Setting the ToT ratio factor of the voxel to the ToT of previous voxel in Cluster Streaming algorithm. Zero factor means ToT of prev. voxel should be larger. 0.5 factor means ToT of prev voxel could be high than the half of the considered voxel

dbscan_clustering

epsilon

min_samples

perform_centroiding_cluster_stream(chunks)

perform_centroiding_dbscan(chunks)

perform_clustering_dbscan(shot, x, y, tof)

The clustering with DBSCAN, which is performed in this function is dependent on the order of the data in rare cases. Therefore, reordering in any means can lead to slightly changed results, which should not be an issue.

Martin Ester, Hans-Peter Kriegel, Jiirg Sander, Xiaowei Xu: A Density Based Algorithm for Discovering Clusters [p. 229-230] (https://www.aaai.org/Papers/KDD/1996/KDD96-037.pdf) A more specific explaination can be found here: https://stats.stackexchange.com/questions/306829/why-is-dbscan-deterministic

process(data)

tot_threshold

Determines which time over threshold values to filter before centroiding

This is useful in reducing the computational time in centroiding and can filter out noise.

triggers_processed

Setting for the number of packets skipped during processing. Every packet_skip packet is processed. This means for a value of 1 every packet is processed. For 2 only every 2nd packet is processed.

class pymepix.processing.logic.centroid_calculator.CentroidCalculatorPooled(number_of_processes=None, *args, **kwargs)

Bases: pymepix.processing.logic.centroid_calculator.CentroidCalculator

Parallelized implementation of CentroidCalculator using mp.Pool for parallelization.

perform_centroiding(chunks)

post_process()

pre_process()

pymepix.processing.logic.centroid_calculator.calculate_centroids_dbscan(chunk, tot_threshold, _tof_scale, epsilon, min_samples, _cent_timewalk_lut)

pymepix.processing.logic.centroid_calculator.calculate_centroids_properties(shot, x, y, tof, tot, labels, _cent_timewalk_lut)

Calculates the properties of the centroids from labeled data points.

ATTENTION! The order of the points can have an impact on the result due to errors in the floating point arithmetics.

Very simple example: arr = np.random.random(100) arr.sum() - np.sort(arr).sum() This example shows that there is a very small difference between the two sums. The inaccuracy of floating point arithmetics can depend on the order of the values. Strongly simplified (3.2 + 3.4) + 2.7 and 3.2 + (3.4 + 2.7) can be unequal for floating point numbers.

Therefore there is no guarantee for strictly equal results. Even after sorting. The error we observed can be about 10^-22 nano seconds.

Currently this is issue exists only for the TOF-column as the other columns are integer-based values.

pymepix.processing.logic.centroid_calculator.perform_clustering_dbscan(shot, x, y, tof, _tof_scale, epsilon, min_samples)

The clustering with DBSCAN, which is performed in this function is dependent on the order of the data in rare cases. Therefore, reordering in any means can lead to slightly changed results, which should not be an issue.

Martin Ester, Hans-Peter Kriegel, Jiirg Sander, Xiaowei Xu: A Density Based Algorithm for Discovering Clusters [p. 229-230] (https://www.aaai.org/Papers/KDD/1996/KDD96-037.pdf) A more specific explaination can be found here: https://stats.stackexchange.com/questions/306829/why-is-dbscan-deterministic

pymepix.processing.logic.packet_processor module

class pymepix.processing.logic.packet_processor.PacketProcessor(handle_events=True, event_window=(0.0, 10000.0), position_offset=(0, 0), orientation=<PixelOrientation.Up: 0>, start_time=0, timewalk_lut=None, *args, **kwargs)

Bases: pymepix.processing.logic.processing_step.ProcessingStep

Class responsible to transform the raw data coming from the timepix directly into an easier processible data format. Takes into account the pixel- and trigger data to calculate toa and tof dimensions.

process(data):

Process data and return the result. To use this class only this method should be used! Use the other methods only for testing or if you are sure about what you are doing

clearBuffers()

correct_global_time(arr, ltime)

event_window

find_events_fast()

find_events_fast_post()

Call this function at the very end of to also have the last two trigger events processed

getBuffers(val_filter=None)

handle_events

Type:noindex

orientPixels(col, row)

Orient the pixels based on Timepix orientation

post_process()

pre_process()

process(data)

process_pixels(pixdata, longtime)

process_trigger1(pixdata, longtime)

process_trigger2(tidtrigdata, longtime)

updateBuffers(val_filter)

class pymepix.processing.logic.packet_processor.PixelOrientation

Bases: enum.IntEnum

Defines how row and col are intepreted in the output

Down = 2

x=-column, y = -row

Left = 1

x=row, y=-column

Right = 3

x=-row, y=column

Up = 0

Up is the default, x=column,y=row

pymepix.processing.logic.processing_step module

class pymepix.processing.logic.processing_step.ProcessingStep(name, parameter_wrapper_class=<class 'pymepix.processing.logic.shared_processing_parameter.SharedProcessingParameter'>)

Bases: pymepix.core.log.Logger, abc.ABC

Representation of one processing step in the pipeline for processing timepix raw data. Implementations are provided by PacketProcessor and CentroidCalculator. To combine those (and possibly other) classes into a pipeline they have to implement this interface. Also provides pre- and post-process implementations which are required for integration in the online processing pipeline (see PipelineCentroidCalculator and PipelinePacketProcessor).

Currently the picture is the following:

• For post processing the CentroidCalculator and the PacketProcessor are used directly
• PipelineCentroidCalculator and PipelinePacketProcessor build on top of CentroidCalculator and PacketProcessor to provide an integration in the existing online processing pipeline for online analysis.

post_process()

pre_process()

process(data)

pymepix.processing.logic.shared_processing_parameter module

class pymepix.processing.logic.shared_processing_parameter.SharedProcessingParameter(value)

Bases: object

Variang of the ProcessingParameter used for sharing among multiple processes. This class has to be used if running with the multiprocessing pipeline to ensure all instances of the processing classes are updated when parameters are changed.

value

exception pymepix.processing.logic.shared_processing_parameter.UnknownParameterTypeException

Bases: Exception

Module contents