Submission

Uploading predictions to Numerai.

0. BaseSubmitter

BaseSubmitter handles all submission logic common to Numerai Classic and Numerai Signals. Under the hood directory logic is handled by BaseIO. Each submittor should inherit from BaseSubmitter and implement the .save_csv method.

---

source

BaseSubmitter

 BaseSubmitter (directory_path:str,
                api:Union[numerapi.numerapi.NumerAPI,numerapi.signalsapi.S
                ignalsAPI])

Basic functionality for submitting to Numerai.

Uses numerapi under the hood. More info: https://numerapi.readthedocs.io/

:param directory_path: Directory to store and read submissions from.

:param api: NumerAPI or SignalsAPI

1. NumeraiClassicSubmitter

For Numerai Classic submissions. Uses NumerAPI under the hood.

Note that using submitters requires a Key object.

---

source

NumeraiClassicSubmitter

 NumeraiClassicSubmitter (directory_path:str, key:numerblox.key.Key,
                          *args, **kwargs)

Submit for Numerai Classic.

:param directory_path: Base directory to save and read prediction files from.

:param key: Key object containing valid credentials for Numerai Classic.

*args, **kwargs will be passed to NumerAPI initialization.

Example usage 1: NumeraiClassicSubmitter

# example 1
# Initialization (Random credentials)
test_dir = "test_sub"
classic_key = Key(pub_id="UFVCTElDX0lE", secret_key="U1VQRVJfU0VDUkVUX0tFWQ==")
num_sub = NumeraiClassicSubmitter(directory_path=test_dir, key=classic_key)
assert num_sub.dir.is_dir()

# Create random dataframe
n_rows = 100
targets = "prediction_mymodel"
test_dataf = pd.DataFrame(np.random.uniform(size=n_rows), columns=[targets])
test_dataf["id"] = [uuid.uuid4() for _ in range(n_rows)]
test_dataf = test_dataf.set_index("id")
test_dataf.head(2)

🔑 Numerai Auth key initialized with pub_id = 'UFVCTElDX0lE' 🔑

No existing directory found at 'test_sub'. Creating directory...

	prediction_mymodel
id
d5285fe7-1e1e-4d77-b71e-ebb760c79570	0.082175
268a957e-89f1-42fc-b247-4a121b727317	0.601661

CSVs can be saved with .save_csv. NumeraiClassicSubmitter will automatically provide checks to make sure that data is saved correctly.

file_name = "test.csv"
num_sub.save_csv(dataf=test_dataf, file_name=file_name, cols=targets)
num_sub.save_csv(dataf=test_dataf, file_name="test2.csv", cols=targets)
pd.read_csv(f"{test_dir}/{file_name}").head(2)

2022-04-02 08:42:01,943 INFO numexpr.utils: Note: NumExpr detected 16 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
2022-04-02 08:42:01,944 INFO numexpr.utils: NumExpr defaulting to 8 threads.

📄 Saving predictions CSV to 'test_sub/test.csv'. 📄

📄 Saving predictions CSV to 'test_sub/test2.csv'. 📄

	id	prediction
0	d5285fe7-1e1e-4d77-b71e-ebb760c79570	0.082175
1	268a957e-89f1-42fc-b247-4a121b727317	0.601661

NumeraiClassicSubmitter also gives you the option to combine multiple predictions csvs that you already created. Prediction will be standardized by default.

---

source

BaseSubmitter.combine_csvs

 BaseSubmitter.combine_csvs (csv_paths:list, aux_cols:list,
                             era_col:str=None, pred_col:str='prediction')

Read in csv files and combine all predictions with a rank mean.

Multi-target predictions will be averaged out.

:param csv_paths: List of full paths to .csv prediction files.

:param aux_cols: [‘id’] for Numerai Classic.

[‘ticker’, ‘last_friday’, ‘data_type’], for example, with Numerai Signals.

:param era_col: Column indicating era (‘era’ or ‘last_friday’).

Will be used for Grouping the rank mean if given. Skip groupby if no era_col provided.

:param pred_col: ‘prediction’ for Numerai Classic and ‘signal’ for Numerai Signals.

combined = num_sub.combine_csvs(["test_sub/test.csv", "test_sub/test2.csv"], aux_cols=['id'])
assert combined.columns == ['prediction']
combined.head(2)

	prediction
id
d5285fe7-1e1e-4d77-b71e-ebb760c79570	0.11
268a957e-89f1-42fc-b247-4a121b727317	0.72

Uncomment to save CSV and upload predictions in one go.

# Full submission
# num_sub.full_submission(dataf=test_dataf, file_name='test.csv', cols=targets, model_name="test")

After a successful submission, contents can be removed to keep a clean environment.

num_sub.remove_base_directory()
assert not os.path.exists(test_dir)

⚠ Deleting directory for 'NumeraiClassicSubmitter' ⚠
Path: '/Users/clepelaars/Desktop/crowdcent/repositories/numerai-blocks/nbs/test_sub'

2. NumeraiSignalsSubmitter

Numerai Signals submissions. Uses SignalsAPI under the hood.

---

source

NumeraiSignalsSubmitter

 NumeraiSignalsSubmitter (directory_path:str, key:numerblox.key.Key,
                          *args, **kwargs)

Submit for Numerai Signals.

:param directory_path: Base directory to save and read prediction files from.

:param key: Key object containing valid credentials for Numerai Signals.

*args, **kwargs will be passed to SignalsAPI initialization.

Example usage 2: NumeraiSignalsSubmitter

Initialization (Random credentials)

test_dir_signals = "test_sub_signals"
signals_key = Key(pub_id="UFVCTElDX0lE", secret_key="U1VQRVJfU0VDUkVUX0tFWQ==")
signals_sub = NumeraiSignalsSubmitter(directory_path=test_dir_signals, key=signals_key)
assert signals_sub.dir.is_dir()

🔑 Numerai Auth key initialized with pub_id = 'UFVCTElDX0lE' 🔑

No existing directory found at 'test_sub_signals'. Creating directory...

def create_random_signals_dataf(n_rows=5000):
    signals_test_dataf = pd.DataFrame(
        np.random.uniform(size=(n_rows, 1)), columns=["signal"]
    )
    signals_test_dataf["ticker"] = [
        "".join(choices(ascii_uppercase, k=4)) for _ in range(n_rows)
    ]
    last_friday = str((datetime.now() + relativedelta(weekday=FR(-1))).date()).replace("-", "")
    signals_test_dataf['last_friday'] = last_friday
    signals_test_dataf['data_type'] = 'live'
    return signals_test_dataf

signals_test_dataf = create_random_signals_dataf()
signals_test_dataf.head(2)

	signal	ticker	last_friday	data_type
0	0.745859	NCCM	20220401	live
1	0.815436	ZGKU	20220401	live

Saving Signals CSV should fail if there is no valid ticker column or if signal has values outside the range \([0...1]\).

Uncomment to save CSV and upload predictions in one go.

# Full Signals submission
# signals_sub.full_submission(dataf=signals_test_dataf, file_name='signals_test.csv', cols=signals_cols, model_name="test")

After a successful submission, contents can be removed to keep a clean environment.

signals_sub.remove_base_directory()
assert not os.path.exists(test_dir_signals)

⚠ Deleting directory for 'NumeraiSignalsSubmitter' ⚠
Path: '/Users/clepelaars/Desktop/crowdcent/repositories/numerai-blocks/nbs/test_sub_signals'

3. NumerBaySubmitter

Wrapper on top of the tournament submitters, submits to NumerBay to fulfill sale orders.

---

source

NumerBaySubmitter

 NumerBaySubmitter (tournament_submitter:Union[__main__.NumeraiClassicSubm
                    itter,__main__.NumeraiSignalsSubmitter],
                    upload_to_numerai:bool=True,
                    numerbay_username:str=None,
                    numerbay_password:str=None)

Submit to NumerBay to fulfill sale orders, in addition to submission to Numerai.

:param tournament_submitter: Base tournament submitter (NumeraiClassicSubmitter or NumeraiSignalsSubmitter). This submitter will use the same directory path. :param upload_to_numerai: Whether to also submit to Numerai using the tournament submitter. Defaults to True, set to False to only upload to NumerBay. :param numerbay_username: NumerBay username :param numerbay_password: NumerBay password

Example usage 3: NumerBaySubmitter

Numerai submissions (existing file)

# numerai_submitter = NumeraiClassicSubmitter(directory_path="/app/notebooks/tmp", key=key)
# nb_submitter = NumerBaySubmitter(tournament_submitter=numerai_submitter, upload_to_numerai=False,
#                                  numerbay_username="someusername", numerbay_password="somepassword")
# nb_submitter.upload_predictions(file_name='upload.csv', model_name='mymodel',
#                                 numerbay_product_full_name='numerai-predictions-myproduct')

Numerai submissions (from NumerFrame)

# numerai_submitter = NumeraiClassicSubmitter(directory_path="/app/notebooks/tmp", key=key)
# nb_submitter = NumerBaySubmitter(tournament_submitter=numerai_submitter, upload_to_numerai=False,
#                                  numerbay_username="someusername", numerbay_password="somepassword")
# nb_submitter.full_submission(dataf, file_name='upload.csv', model_name='mymodel',
#                              numerbay_product_full_name='numerai-predictions-myproduct',
#                              cols='some_pred_col')

Signals submissions (existing file)

# signals_submitter = NumeraiSignalsSubmitter(directory_path="/app/notebooks/tmp", key=key)
# nb_submitter = NumerBaySubmitter(tournament_submitter=signals_submitter, upload_to_numerai=False,
#                                  numerbay_username="someusername", numerbay_password="somepassword")
# nb_submitter.upload_predictions(file_name='upload-signals.csv', model_name='mymodel',
#                                 numerbay_product_full_name='signals-predictions-myproduct')

Signals submissions (from NumerFrame)

# signals_submitter = NumeraiSignalsSubmitter(directory_path="/app/notebooks/tmp", key=key)
# nb_submitter = NumerBaySubmitter(tournament_submitter=signals_submitter, upload_to_numerai=False,
#                                  numerbay_username="someusername", numerbay_password="somepassword")
# nb_submitter.full_submission(dataf, file_name='upload-signals.csv', model_name='mymodel',
#                              numerbay_product_full_name='signals-predictions-myproduct',
#                              cols=['bloomberg_ticker', 'friday_date', 'data_type', 'signal'])

---