Pangeo Weekly Meeting Minutes

https://columbiauniversity.zoom.us/j/6902819781

2019-10-09 1600 GMT (9am PDT, 12pm EDT)

Name / Institution / @GitHub handle

60 Second Updates:

Agenda Items

2019-10-02 2000 GMT (1pm PDT, 4pm EDT)

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

Short term coordination:


2019-09-25 1600 GMT (9am PDT, 12pm EDT)

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

Short term coordination:



2019-09-18 2000 GMT (1pm PDT, 4pm EDT)


Video Conference Link: https://whereby.com/pangeo

Hello and welcome to the team meeting!
Please sign in below so we know who was here:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

Short term coordination:

2019-09-11 2000 GMT (1pm PDT, 4pm EDT)


Video Conference Link: https://whereby.com/pangeo
Hello and welcome to the team meeting!
Please sign in below so we know who was here:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

Short term coordination:

2019-09-04 2000 GMT (1pm PDT, 4pm EDT)

Zoom or Appear today?


Video Conference Link: https://whereby.com/pangeo
Hello and welcome to the team meeting!
Please sign in below so we know who was here:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

Short term coordination:

2019-08-28 1600 GMT (9am PDT, 12pm EDT)


Video Conference Link: https://appear.in/pangeo

Hello and welcome to the team meeting!
Please sign in below so we know who was here:

Name / Institution / @GitHub handle

60 second updates

2019-08-14 1600 GMT (9am PDT, 12pm EDT)


Video Conference Link: https://appear.in/pangeo

Hello and welcome to the team meeting!
Please sign in below so we know who was here:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

Short term coordination:

.

2019-08-07 2000 GMT (1pm PDT, 4pm EDT)


Video Conference Link: https://appear.in/pangeo

Hello and welcome to the team meeting!
Please sign in below so we know who was here: ········

Name / Institution / @GitHub handle

60 Second Updates

Agenda items:

2019-07-31 1600 GMT (9am PDT, 12pm EDT)


Video Conference Link: https://appear.in/pangeo

Hello and welcome to the team meeting!
Please sign in below so we know who was here:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

Short term coordination:

2019-07-24 2000 GMT (1pm PDT, 4pm EDT)


Video Conference Link: https://appear.in/pangeo
Hello and welcome to the team meeting!
Please sign in below so we know who was here:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

Short term coordination:

2019-07-17 1600 GMT (9am PDT, 12pm EDT)


Video Conference Link: https://appear.in/pangeo

Hello and welcome to the team meeting!

Please sign in below so we know who was here:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

Short term coordination:

[Special SciPy 2019 BOF edition] 2019-07-11 1800 GMT (11am PDT, 2pm EDT)  

71 people in the room!

Joe Hamman kicks off BOF with 5 minute intro of slides.

One minute intro from folks in the room:

Institutions represented:

Capital One Center for Machine Learning, Brookhaven, USACE, Argonne, Northeastern Kostas, NCAR, NASA Goddard, GOMRI, Chan-Zuckerberg Initative, MBARI, Oceanweather, Shell Oil, Quantum Spatial , EDF Renewable Energy, LDGO, Microsoft, IOOS, Texas Water Development Board, UCSB, UC Boulder, U Congo ,  ?? Marine, Monash University, Descarte Labs,  NASA Ames, NVIDIA, RENCI

Fields of interest:

Machine Learning, Cell Biology, Oceanography, Meteorology, Physics, Homeland Security, Geospatial Analysis, Hydrology, Remote Sensing, Microscopy,Biomedical imaging , Heathcare, Precision Agriculture, Groundwater Modeling, Astronomical Data, Hydrology

Ryan give a live demo of Pangeo on the Cloud on ocean.pangeo.io

Q: Do you need to run on Cloud, what do you do on HPC?

A: See Anderson’s talk later

Q: Do you need to use Zarr

A: No, Zarr works well on cloud, but you can use anything that xarray can read

Q: How do custom environments work?

A: Different JH use different default environments

Q: How does metering/cost/tracking for users work?

A:  Anyone can access data with S3, also working with NASA EarthDataLogin.

Q: What about GCM data?

A: Pangeo folks expect that providers will see the value of moving their data to the Cloud, and do it themselves.  

Q: You can we can just try this, how does that work?

A: You can use the pangeo binder right now with your own repo.  The pangeo binder is like mybinder.org but can launch dask clusters,  and has a bit more memory

Q: Seems like we heard about a lot of different fields that want Pangeo, what should we do?

A: You can set up your own using the Pangeo deployment guides. Some pangeo secrets are

2019-07-10 2000 GMT (1pm PDT, 4pm EDT)


Video Conference Link: https://appear.in/pangeo

Hello and welcome to the team meeting!

Please sign in below so we know who was here:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

Short term coordination:

2019-07-03 1600 GMT (9am PDT, 12pm EDT)


Video Conference Link: https://appear.in/pangeo

Hello and welcome to the team meeting!

Please sign in below so we know who was here:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

Short term coordination:

2019-06-26 2000 GMT (1pm PDT, 4pm EDT)


Video Conference Link: https://appear.in/pangeo

Hello and welcome to the team meeting!
Please sign in below so we know who was here:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

2019-06-19 1600 GMT (9am PDT, 12pm EDT)


Conference Link: https://appear.in/pangeo

Hello and welcome to the Pangeo team meeting!
Please sign in below so we know who was here:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

Short term coordination:

2019-06-12  (4pm EDT)


Hello and welcome! If you are joining the team video meeting, sign in below so we know who was here:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda Items:

2019-06-05  (9am PDT, 12pm EDT)

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda Items:

Short Term Coordination

2019-05-29 1600 GMT (1pm PDT, 4pm EDT)

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda Items:

2019-05-22 1600 GMT (9am PDT, 12pm EDT)


Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda Items:

  1. Rob: Working Group Update
  1. Exemplary notebooks exist, build more inclusive notebooks
  2. Agenda for first WG meeting: identify where Pangeo is needed; don’t reinvent
  3. Joe: can point to Zero to Pangeo, Zero to Jupyterhub
  1. Pangeo meeting in Seattle
  1. Jim Bednar will be at ESIP, with a PyViz overview talk
  2. Reach out to Chelle Gentemann
  1. Sean’s issues: eksctl vs kops

2019-05-15 2000 GMT (1pm PDT, 4pm EDT)

Due to expected attendance < 15, back to appear.in.

Conference link:  https://appear.in/pangeo

Welcome to the Team Meeting.

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

Name / Institution / @GitHub handle

60 Second Updates:

Working on intake dependencies and import time

Agenda items:

Short term coordination:


2019-05-08 1600 GMT (9am PDT, 12pm EDT)

Video Conference Link:  Zoom

Join Zoom Meeting

https://columbiauniversity.zoom.us/j/903370578

One tap mobile

+16468769923,,903370578# US (New York)
+16699006833,,903370578# US (San Jose)

Dial by your location

+1 646 876 9923 US (New York)
+1 669 900 6833 US (San Jose)

Meeting ID: 903 370 578

Find your local number: https://zoom.us/u/abcuAxbTMF

Welcome to the Team Meeting.

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

Short term coordination:

2019-05-01 2000 GMT (1pm PDT, 4pm EDT)
Instead of appear.in, this week we are using  Zoom Video + Audio Meeting from any device:
https://columbiauniversity.zoom.us/j/535776170

One tap mobile:
+16468769923,,535776170# US (New York)
+16699006833,,535776170# US (San Jose)

Dial by your location:
+1 646 876 9923 US (New York)
+1 669 900 6833 US (San Jose)

Meeting ID: 535 776 170

Find your local number: https://zoom.us/u/abcuAxbTMF

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

Short term coordination:

2019-04-24 1600 GMT (9am PDT, 12pm EDT)

Video Conference Link:  https://appear.in/pangeo

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

Name / Institution / GitHub handle

60 Second Updates:

Agenda items

Short term coordination:

2019-04-17 8pm GMT (1pm PDT, 4pm EDT)


Video Conference Link: https://appear.in/pangeo

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

60 Second Updates:

Agenda items

Short term coordination:

2019-04-10 (9am PDT, 12pm EDT)


Video Conference Link: https://appear.in/pangeo

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

60 Second Updates:

Will run with dataset with same chunking, compression and number of CPUs):

Struggling a bit with best tool for rechunking (nco, nccopy, python w/dask)

GSoC proposals closed yesterday, have  a great candidate for project to build ncView-like tool of future with pyviz/panel app

Agenda items

2019-04-3 8pm GMT (1pm PDT, 4pm EDT)


Video Conference Link: https://appear.in/pangeo

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

60 Second Updates:

Agenda items:

Just checks notebooks for errors (doesn’t verify content)

Short term coordination:

2019-03-27 16:00 GMT (9am PT, 12pm EST)

Video Conference Link: https://appear.in/pangeo

Welcome to the Team Meeting.

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

60 Second Updates:

Agenda items
Let's collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them already.

2019-03-20 8pm GMT (1pm PDT, 4pm EDT)


Video Conference Link: https://appear.in/pangeo

Welcome to the Team Meeting.

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

60 Second Updates:

Agenda items
Let's collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them already.


2019-03-13 4pm GMT (9am PDT, 12pm EDT)

Video Conference Link: https://appear.in/pangeo

Welcome to the Team Meeting.

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

60 Second Updates:

Reports and celebrations

Things people should know about.

Agenda items
Let's collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them already.


2019-03-06

In attendance:

Quick Reports:

   

Status Check on Larger Issues:

Coordination Items:

2019-02-27

In attendance:        

Quick Reports:

Longer Discussions

Short-Term Coordination:

2019-02-20

In attendance:        

Quick Reports:

Longer Discussions


Short-Term Coordination:

2019-02-13

https://columbiauniversity.zoom.us/j/666242279

In attendance:

Quick Reports

Status check on larger issues

https://docs.google.com/document/d/1Of__qhrAPES9mhnnby9SDRsKY2qV_mFrgLhr1CGgciI/edit?usp=sharing 

Short term coordination

2019-02-06

In attendance:

Quick Reports

Status check on larger issues

Short-term Coordination:

2019-01-30

In attendance:

Quick Reports

Status check on larger issues

Short term coordination

2019-01-23

In attendance:

Quick Reports

Status check on larger issues

Short term coordination

2019-01-16

In attendance: Joe H., Ryan A, Kevin P, Lindsey H., Rachel P, Anderson B., Julia K, Rob F

Quick Reports

Status check on larger issues

Short term coordination

2019-01-09

In attendance:  Joe H., Jim B. Matt R., Noah B., Raphael D., Ryan A., Rich S., Alando, Ariel R.

Quick Reports

Status check on larger issues

Action Items:

2018-12-19

In attendance: Raphael, Anderson, Kevin, Joe, Matt, Jacob, Pete

Quick Reports

Status check on larger issues

Short term coordination

2018-12-05

In attendance: Matt, Niall, Rich, Raphael,  Anderson, Kevin, Joe

Quick Reports

 Should this be in xarray or a stand-alone package?

Status check on larger issues

Short term coordination

Rich works with Joe to improve best practices for dask-jobqueue.   (e.g. if folks want to speed up their calculations, what considerations are there based on the type of job and type of nodes on their HPC?)   Joe says his HPC jobs are often I/O bound (e.g reading from NetCDF), so doesn’t use too many threads per worker.  But if reading from something like zarr that is thread-safe, might go with more threads per worker.   Also usually looks at how many cpus/node are available and submits jobs with one node per job.

2018-11-28

In attendance: Joe, Ryan, Kevin, Anderson, Raphael, Derek, Matt

Quick Reports

Status check on larger issues

Short term coordination

2018-11-21

In attendance: Matt, Rich, Raphael, Joe, Ryan

Quick Reports

Status check on larger issues

Short term coordination

2018-11-14

In attendance: Anderson, Matt, Rich, Jacob, Raphael, Kevin Tyle, Joe

Quick Reports

Status check on larger issues

Short term coordination

2018-11-07

In attendance: Joe, Ryan, Matt, Guillaume

Quick Reports

Status check on larger issues

Short term coordination

2018-10-24

In attendance: Matt, Anderson, Ryan, Raphael, Derek, Rich

Quick Reports

Status check on larger issues

Short term coordination

2018-10-17

In attendance: Ryan, Raphael, Joe, Kevin, Matt, Nick Wayland, Anderson, Rich Signell

Quick Reports

Status check on larger issues

Short term coordination

2018-10-10

In attendance: Ryan Abernathey, Matt Rocklin, Derek Ludwig, Anderson B

Quick Reports

Status check on larger issues

Short term coordination

2018-10-03

In attendance: Raphael, Anderson, Guillaume, Rich, Derek, Joe, Matt

Quick Reports

Status check on larger issues

Short term coordination

2018-09-26

In attendance: Ryan Abernathey, Joe Hamman, Matt Rocklin, Rich Signell, Derek Ludwig, Niall Robinson, Jacob Tomlinson

Quick reports

Status Check on larger issues

Short term coordination


Meeting Template:

YYYY-MM-DD 1600 GMT (9am PDT, 12pm EDT)
YYYY-MM-DD 2000 GMT (1pm PDT, 4pm EDT)


Video Conference Link: https://whereby.com/pangeo
Video Conference Link:  Zoom

Join Zoom Meeting:
https://columbiauniversity.zoom.us/j/535776170

One tap mobile:
+16468769923,,535776170# US (New York)
+16699006833,,535776170# US (San Jose)

Dial by your location:
+1 646 876 9923 US (New York)
+1 669 900 6833 US (San Jose)

Meeting ID: 535 776 170

Find your local number: https://zoom.us/u/abcuAxbTMF

Hello and welcome to the team meeting!
Please sign in below so we know who was here:

Name / Institution / @GitHub handle

60 Second Updates:

Agenda items:

Short term coordination: