processor
core.processor
Data processing module.
Classes
| Name | Description |
|---|---|
| DataProcessor | Data processing system. |
DataProcessor
core.processor.DataProcessor(self)Data processing system.
Methods
| Name | Description |
|---|---|
| deduplicate_columns | Remove duplicate columns. |
| optimize_memory | Optimize DataFrame memory usage. |
| process_systems_data | Process systems data. |
deduplicate_columns
core.processor.DataProcessor.deduplicate_columns(df)Remove duplicate columns.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| df | pd.DataFrame | Input DataFrame with potential duplicate columns | required |
Returns
| Name | Type | Description |
|---|---|---|
| Tuple[pd.DataFrame, Dict] | Deduplicated DataFrame and mapping of removed columns |
optimize_memory
core.processor.DataProcessor.optimize_memory(df)Optimize DataFrame memory usage.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| df | pd.DataFrame | Input DataFrame to optimize | required |
Returns
| Name | Type | Description |
|---|---|---|
| pd.DataFrame | Memory-optimized DataFrame |
process_systems_data
core.processor.DataProcessor.process_systems_data(df, optimize_memory=True)Process systems data.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| df | pd.DataFrame | Input DataFrame to process | required |
| optimize_memory | bool | Whether to optimize memory usage, by default True | True |
Returns
| Name | Type | Description |
|---|---|---|
| pd.DataFrame | Processed DataFrame |