How do you create data objects that can handle missing data?

1 Why OOD for data objects?

OOD is a software development paradigm that structures code into objects, which have attributes and methods that define their behavior and interactions. OOD can help you create data objects that are encapsulated, abstract, polymorphic, and able to inherit attributes and methods. This makes them easier to maintain, debug, understand, communicate, adapt, extend, reuse, and modify.

2 How to handle missing data in OOD?

Missing data can arise from a variety of sources, such as incomplete or corrupted data sources, user input errors, or intentional omissions. If not handled properly in data objects, missing data can adversely impact the quality and reliability of your data analysis and modeling. To handle missing data in OOD, you can either ignore it and proceed as if it does not exist; however, this approach can lead to inaccurate or biased results. Alternatively, you can impute missing data with some reasonable values, such as the mean, median, mode, or a constant. This approach is more complex and slow but can improve the quality and reliability of your data analysis and modeling or at least avoid errors and exceptions.

3 How to design data objects that can ignore missing data?

To design data objects that can ignore missing data, you need to ensure that your data objects can validate, filter, and handle missing values. You can use constructors, setters, validators, methods, properties, filters, try-except blocks, error handlers, or logs to perform this task. For example, you can use the isna() , dropna() , and try and except keywords in pandas and Python respectively to check for missing values, remove rows or columns with missing values, and catch and handle a ValueError exception raised by a missing value.

4 How to design data objects that can impute missing data?

Designing data objects that can impute missing data requires ensuring they can identify the missing values and their patterns in their input data, replace the missing values with reasonable values in their output data, and evaluate the impact and quality of the imputation on their output data. To do this, you can use methods, properties, or indicators to identify the missing values; use methods, properties, or imputers to replace them; and use methods, properties, or metrics to evaluate the imputation. For example, you can use the isna() , sum() , and fillna() methods in pandas to count and fill the missing values in a DataFrame object; and use the describe() method to summarize the statistics of a DataFrame object before and after imputation.

5 How to choose between ignoring and imputing missing data?

When deciding whether to ignore or impute missing data, there is no one-size-fits-all answer as it depends on the nature and context of your data and your analysis goals. Generally speaking, you should ignore missing data if it is random, insignificant, or irrelevant to your analysis, or if it does not affect the performance or accuracy of your data objects or models. On the other hand, you should impute missing data if it is systematic, significant, or relevant to your analysis, or if it affects the performance or accuracy of your data objects or models. Ultimately, you should compare and test different methods of ignoring and imputing missing data and choose the one that best suits your data and your analysis.

6 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?