Without sufficient data and suitable features, no matter how powerful the model structure is, it will not be able to get satisfactory outputs. As a classic saying goes, "garbage in, garbage out". For a machine Xi problem, data and features often determine the upper limit of the result, and the selection and optimization of models and algorithms are gradually approaching this upper limit.
Feature engineering, as the name suggests, is a series of engineering processes on raw data, distilling it into features that can be used as input for algorithms and models. Essentially, feature engineering is the process of representing and presenting data. In practice, feature engineering aims to remove impurities and redundancies from the original data, and design more efficient features to characterize the relationship between the solved problem and the model.
There are two common types of data that engineers typically deal with.
1) Structured data. A structured data type can be thought of as a table in a relational database, with each column clearly defined, including two basic types: numeric and categoricalEach row of data represents information for a sample.
2) Unstructured data. Unstructured data mainly includes text, images, audio, and **data, which contains information that cannot be represented by a simple numerical value, and there is no clear category definition, and the size of each piece of data varies.
In order to eliminate the dimensional influence between data features, we need to normalize the features so that they are comparable between different indicators. For example, to analyze the impact of a person's height and weight on health, if meters (m) and kilograms (kg) are used as units, then the height characteristics are concentrated at 16～1.In the range of 8 m, the weight characteristics will be in the range of 50 to 100 kg, and the results of the analysis will obviously tend to favor the weight characteristics with large numerical differences. In order to get more accurate results, it is necessary to normalize the characteristics so that each indicator is in the same numerical order for analysis.
AI assistant creation season