You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Join us on [](https://tiny.ydata.ai/dcai-ydata-synthetic)
12
12
13
13
# YData Synthetic
14
-
A package to generate synthetic tabular and time-series data leveraging the state of the art generative models.
15
-
16
-
## 🎊 The exciting features:
17
-
> These are must try features when it comes to synthetic data generation:
18
-
> - A new streamlit app that delivers the synthetic data generation experience with a UI interface. A low code experience for the quick generation of synthetic data
19
-
> - A new fast synthetic data generation model based on Gaussian Mixture. So you can quickstart in the world of synthetic data generation without the need for a GPU.
20
-
> - A conditional architecture for tabular data: CTGAN, which will make the process of synthetic data generation easier and with higher quality!
21
-
14
+
`YData-Synthetic` is an open-source package developed in 2020 with the primary goal of educating users about generative models for synthetic data generation.
15
+
Designed as a collection of models, it was intended for exploratory studies and educational purposes.
16
+
However, it was not optimized for the quality, performance, and scalability needs typically required by organizations.
17
+
18
+
!!! note "Update"
19
+
Even though the journey was fun, and we have learned a lot from the community it is now time to upgrade `ydata-synthetic`.
20
+
Heading towards the future of synthetic data generation we recommend users to transition to `ydata-sdk`, which provides a superior experience with enhanced performance,
21
+
precision, and ease of use, making it the preferred tool for synthetic data generation and a perfect introduction to Generative AI.
22
+
22
23
## Synthetic data
23
24
### What is synthetic data?
24
25
Synthetic data is artificially generated data that is not collected from real world events. It replicates the statistical components of real data without containing any identifiable information, ensuring individuals' privacy.
@@ -32,68 +33,50 @@ Synthetic data can be used for many applications:
32
33
33
34
> **Looking for an end-to-end solution to Synthetic Data Generation?**<br>
34
35
> [YData Fabric](https://ydata.ai/products/synthetic_data) enables the generation of high-quality datasets within a full UI experience, from data preparation to synthetic data generation and evaluation.<br>
35
-
> Check out the [Community Version](https://ydata.ai/ydata-fabric-free-trial).
36
-
37
-
38
-
# ydata-synthetic
39
-
This repository contains material related with architectures and models for synthetic data, from Generative Adversarial Networks (GANs) to Gaussian Mixtures.
40
-
The repo includes a full ecosystem for synthetic data generation, that includes different models for the generation of synthetic structure data and time-series.
41
-
All the Deep Learning models are implemented leveraging Tensorflow 2.0.
42
-
Several example Jupyter Notebooks and Python scripts are included, to show how to use the different architectures.
43
-
44
-
Are you ready to learn more about synthetic data and the bext-practices for synthetic data generation?
45
-
46
-
## Quickstart
47
-
The source code is currently hosted on GitHub at: https://github.com/ydataai/ydata-synthetic
48
-
49
-
Binary installers for the latest released version are available at the [Python Package Index (PyPI).](https://pypi.org/project/ydata-synthetic/)
50
-
```commandline
51
-
pip install ydata-synthetic
52
-
```
36
+
> Check out the [Community Version](https://ydata.ai/register).
53
37
54
-
### The UI guide for synthetic data generation
55
38
56
-
YData synthetic has now a UI interface to guide you through the steps and inputs to generate structure tabular data.
57
-
The streamlit app is available form *v1.0.0* onwards, and supports the following flows:
58
-
- Train a synthesizer model
59
-
- Generate & profile synthetic data samples
39
+
## ydata-synthetic to ydata-sdk
40
+
With the upcoming update of `ydata-synthetic`to `ydata-sdk`, users will now have access to a single API that automatically selects and optimizes
41
+
the best generative model for their data. This streamlined approach eliminates the need to choose between
42
+
various models manually, as the API intelligently identifies the optimal model based on the specific dataset and use case.
60
43
61
-
#### Installation
44
+
Instead of having to manually select from models such as:
62
45
63
-
```commandline
64
-
pip install ydata-synthetic[streamlit]
65
-
```
66
-
#### Quickstart
67
-
Use the code snippet below in a python file (Jupyter Notebooks are not supported):
-[TimeGAN](https://papers.nips.cc/paper/2019/file/c9efe5f26cd17ba6216bbe2a7d26d490-Paper.pdf) (specifically for *time-series* data)
55
+
-[DoppelGANger](https://dl.acm.org/doi/pdf/10.1145/3419394.3423643) (specifically for *time-series* data)
70
56
71
-
streamlit_app.run()
72
-
```
57
+
The new API handles model selection automatically, optimizing for the best performance in fidelity, utility, and privacy.
58
+
This significantly simplifies the synthetic data generation process, ensuring that users get the highest quality output without
59
+
the need for manual intervention and tiring hyperparameter tuning.
73
60
74
-
Or use the file streamlit_app.py that can be found in the [examples folder](https://github.com/ydataai/ydata-synthetic/tree/master/examples/streamlit_app.py).
61
+
Are you ready to learn more about synthetic data and the best-practices for synthetic data generation?
62
+
For more materials on [synthetic data generation with Python see the documentation](https://docs.fabric.ydata.ai/latest/sdk/).
75
63
64
+
## Quickstart
65
+
Binary installers for the latest released version are available at the [Python Package Index (PyPI).](https://pypi.org/project/ydata-sdk/)
76
66
```commandline
77
-
python -m streamlit_app
67
+
pip install ydata-sdk
78
68
```
79
69
80
-
The below models are supported:
81
-
- CGAN
82
-
- WGAN
83
-
- WGANGP
84
-
- DRAGAN
85
-
- CRAMER
86
-
- CTGAN
70
+
### The UI guide for synthetic data generation
87
71
88
-
[](https://youtu.be/ep0PhwsFx0A)
72
+
YData Fabric offers an UI interface to guide you through the steps and inputs to generate structure data.
73
+
You can experiment today with [YData Fabric by registering the Community version](https://ydata.ai/register).
89
74
90
75
### Examples
91
76
Here you can find usage examples of the package and models to synthesize tabular data.
92
-
- Fast tabular data synthesis on adult census income dataset [](https://colab.research.google.com/github/ydataai/ydata-synthetic/blob/master/examples/regular/models/Fast_Adult_Census_Income_Data.ipynb)
93
-
- Tabular synthetic data generation with CTGAN on adult census income dataset [](https://colab.research.google.com/github/ydataai/ydata-synthetic/blob/master/examples/regular/models/CTGAN_Adult_Census_Income_Data.ipynb)
94
-
- Time Series synthetic data generation with TimeGAN on stock dataset [](https://colab.research.google.com/github/ydataai/ydata-synthetic/blob/master/examples/timeseries/TimeGAN_Synthetic_stock_data.ipynb)
95
-
- Time Series synthetic data generation with DoppelGANger on FCC MBA dataset [](https://colab.research.google.com/github/ydataai/ydata-synthetic/blob/master/examples/timeseries/DoppelGANger_FCC_MBA_Dataset.ipynb)
96
-
- More examples are continuously added and can be found in `/examples` directory.
77
+
- Tabular [synthetic data generation on Titanic Kaggle dataset](https://github.com/ydataai/ydata-sdk/blob/main/examples/synthesizers/regular_quickstart.py)
78
+
- Time Series [synthetic data generation]('https://github.com/ydataai/ydata-sdk/blob/main/examples/synthesizers/time_series_quickstart.py')
79
+
- More examples are continuously added and can be found in [examples directory](https://github.com/ydataai/ydata-sdk/tree/main/examples).
97
80
98
81
### Datasets for you to experiment
99
82
Here are some example datasets for you to try with the synthesizers:
@@ -108,7 +91,7 @@ Here are some example datasets for you to try with the synthesizers:
108
91
109
92
## Project Resources
110
93
111
-
In this repository you can find the several GAN architectures that are used to create synthesizers:
94
+
Find below useful literature of how to generate synthetic data and available generative models:
112
95
113
96
### Tabular data
114
97
-[GAN](https://arxiv.org/abs/1406.2661)
@@ -125,11 +108,6 @@ In this repository you can find the several GAN architectures that are used to c
We are open to collaboration! If you want to start contributing you only need to:
130
-
1. Search for an issue in which you would like to work. Issues for newcomers are labeled with good first issue.
131
-
2. Create a PR solving the issue.
132
-
3. We would review every PRs and either accept or ask for revisions.
133
111
134
112
## Support
135
113
For support in using this library, please join our Discord server. Our Discord community is very friendly and great about quickly answering questions about the use and development of the library. [Click here to join our Discord community!](https://tiny.ydata.ai/dcai-ydata-synthetic)
`ydata-synthetic` is available through PyPi, allowing an easy process of installation and integration with the data science programing environments (Google Colab, Jupyter Notebooks, Visual Studio Code, PyCharm) and stack (`pandas`, `numpy`, `scikit-learn`).
2
+
`ydata-sdk` is available through PyPi, allowing an easy process of installation and integration with the data science programing environments (Google Colab, Jupyter Notebooks, Visual Studio Code, PyCharm) and stack (`pandas`, `numpy`, `scikit-learn`).
3
3
4
4
##Installing the package
5
-
Currently, the package supports **python versions over 3.9**, and can be installed in Windows, Linux or MacOS operating systems.
5
+
Currently, the package supports **python versions over 3.9 and up-to python 3.12**, and can be installed in Windows, Linux or MacOS operating systems.
6
6
7
7
Prior to the package installation, it is recommended the creation of a virtual or `conda` environment:
8
8
9
9
=== "conda"
10
10
``` commandline
11
-
conda create -n synth-env python=3.10
11
+
conda create -n synth-env python=3.12
12
12
conda activate synth-env
13
13
```
14
14
15
-
The above command creates and activates a new environment called "synth-env" with Python version 3.10.X. In the new environment, you can then install `ydata-synthetic`:
15
+
The above command creates and activates a new environment called "synth-env" with Python version 3.12.X. In the new environment, you can then install `ydata-sdk`:
@@ -27,20 +27,7 @@ The above command creates and activates a new environment called "synth-env" wit
27
27
To install inside a Google Colab notebook, you can use the following:
28
28
29
29
```commandline
30
-
!pip install ydata-synthetic==1.1.0
30
+
!pip install ydata-sdk
31
31
```
32
32
33
-
Make sure your Google Colab is running Python versions `>=3.9, <3.11`. Learn how to configure Python versions on Google Colab [here](https://stackoverflow.com/questions/68657341/how-can-i-update-google-colabs-python-version/68658479#68658479).
34
-
35
-
36
-
## Installing the Streamlit App
37
-
Since version 1.0.0, the `ydata-synthetic` includes a GUI experience provided by a Streamlit app. The UI supports the data synthesization process from reading the data to profiling the synthetic data generation, and can be installed as follows:
38
-
39
-
```commandline
40
-
pip install "ydata-synthetic[streamlit]"
41
-
```
42
-
43
-
Note that Jupyter or Colab Notebooks are not yet supported, so use it in your Python environment.
Make sure your Google Colab is running Python versions `>=3.9, <=3.12`. Learn how to configure Python versions on Google Colab [here](https://stackoverflow.com/questions/68657341/how-can-i-update-google-colabs-python-version/68658479#68658479).
0 commit comments