The workshop will take place in May 7-8, 2018 at:

New York University
Center for Cosmology and Astro-Particle Physics
Room 802
726 Broadway
New York, NY 10003

Goals of Workshop

Day 1:

Metrics: code up the metric we want to use as the Kaggle decision metric and test on 2010 Challenge data

Validation:  complete checks on individual models, identify remaining issues and  decide on the training data.

Day 2:

Validation: test a few classifiers, identify sample wide classification issues  and produce mock data set to be sent to Kaggle

Metrics: finishing coding up a few additional metrics that are of interest and apply them to the mock data set

Agenda for Monday – May 7th, 2018

09:00 – 09:15:    Welcome and meeting goals – Renee Hlozek

09:15 – 09:30:    Metrics status update and todo list – Renee Hlozek

09:30 – 09:45:    Updates on deployment platforms – Emille Ishida

Room 1: Validation

09:45 – 10:00:     Validation status updates and todo list – Gautham Narayan

10:00 – 11:00:      (Brainstorm) Identify problematic models and separate tasks

11:00 – 12:00:      (Hack) Implement tests and identify problems

12:00 – 14:00:     Lunch

14:00 – 16:30:      (Hack) Implement tests and identify remaining problems

16:30 – 18:00:      (Hack/Brainstorm) Decide on a recipe for the training set, and implement it

Room 2: Metrics

09:45 – 11:00:     (Brainstorm) Identify the Kaggle metric –  full light-curve, supervised learning scenario

11:00 – 12:00:       (Hack)   Code the chosen metric  and apply it to results from the 2010 Challenge data

12:00 – 14:00:      Lunch

14:00 – 16:30:       (Hack)   Code up the chosen metric  and apply it to results from the 2010 Challenge data

16:30 – 18:00:        (Hack/Brainstorm) Identify a few other interesting  metrics

Agenda for Tuesday – May 8th, 2018

9:00 – 09:30:      Metrics:  Status update and to do list – TBA

Room 1:  Validation

09:30 – 10:00:      Validation: Status update and to do list – TBA

10:00 – 11:00:       (Brainstorm) Identify simple classifiers   and separate tasks

11:00 – 12:00:       (Hack) Implement the classifiers and check for data issues that might result in obvious classification

12:00 – 14:00:  Lunch

14:00 – 16:00:       (Hack) Implement the classifiers and check for obviously classified models and identify issues

Room 2: Metrics

10:30 – 12:00:        (Brainstorm) Identify metrics options of a couple of different science cases

12:00 – 14:00:   Lunch

14:00 – 16:00:        (Hack) Implement the metrics cases considered desirable

16:00 – 17:00:       (Hack) Apply the Kaggle metrics to classification results

17:00 – 18:00:       Summary and next steps – Renee Hlozek