RoadMap - April 2016
Goals of the Roadmap
We'd like to capture the current state of GeoMesa in terms of major work items, desired features, and crazy new ideas! Feel free to add notes or updates, etc. Please don't delete ideas but make them strickthrough unless its really just not needed. If you do add a new item, try to flush it out with a good description so that its useful to other people who may pick up your idea in the future.
Tooling & Automation
Goal/Area | Description | POC/Lead | JIRA | Status |
---|---|---|---|---|
Installation automation | - AWS, Cloudera, Hortonworks, general | |||
Data modeling tooling |
| |||
JEMA like workflow tooling | - hosted analytics - YARN app for load balancing workflows - parallelization - data and analytic hosting platform? | |||
Amazon Marketplace? | ||||
Tools for Hbase, Cassandra, Dynamo, etc | We have accumulo and kafka. maybe add some basic tools for the other ones | |||
Push Button Deploys | Everything |
Indexing, Performance, Optimization
Goal/Area | Description | POC/Lead | JIRA | Status |
---|---|---|---|---|
Pluggable SFCs | S2 evaluation, Hilbert, etc. The Nasa thing Andrew Annex (Deactivated) please put it in here, metro hash? http://healpix.sourceforge.net/ http://blog.christianperone.com/2015/08/googles-s2-geometry-on-the-sphere-cells-and-hilbert-curve/
| |||
Interval index? | start and end times
| |||
Cost-based Optimization | - summary stats - vacuum analyze | |||
Performance | Move these out into separate line items as we flush them out - Aggregate/batch simple features in iterator like we do for bin queries - don't double store feature ID (once in row, once in simple feature) - would require changes to deserializationn and might complicate batching simple features - automatically sample results for WMS queries based on expected result size
- use lazy deserialization in client
| |||
Spatial Joins | ||||
Pan and zoom caching | ||||
Configurable z shards | - currently hard-coded at four shards - number of ranges we compute might need to be based on the shards so we don't create too many | |||
Complex Feature Support |
Integration Efforts
Integration of GeoMesa with new Open Source tools, databases, platforms, etc
Product | Description/Ideas | POC/Lead | JIRA | Status |
---|---|---|---|---|
DynamoDB | ||||
Cassandra | ||||
Kudu/Impala/Parquest | ||||
Jupyter/iPython/R notebook | ||||
Graphite | Metrics/graphite and predefined dashboards - streaming metrics | |||
Nifi | - flow definitions for canonical setups | |||
Hive | (see Hortonworks visual sql builder) | |||
Pig | ||||
Cloud native storage | You can't afford to use AWS for petabytes? S3 Storage - store binary files, fairly static Use case: Using weather stuff Openstack?
|
Support/Customer Relations/Compliance
Item | Description/Ideas | POC/Lead | JIRA | Status |
---|---|---|---|---|
Avro WFS MimeType plugin | need to open source | GEOMESA-840 | ||
Documentation | What do we need additional Documentation for?
Should we open up Confluence? What about documentation that changes or new tutorials? | |||
FAQ page | for common errors, configuration, etc | |||
Data defined ageoff | Need to be able to age off data based on some criteria in the SFT or visibilities | GEOMESA-899 | ||
Query Auditing | Need to be able to audit all queries against a GeoServer WFS/WMS/WPS/REST/ETC
| Seapy, Thomas, Chris, Jim | GEOMESA-1173 | |
Accumulo 1.7 | Support for Accumulo 1.7 |
Visualization
Stealth
- polish
- open source
- modularity
New Features, Capabilities, & Brainstorms
Area | Description/Ideas | POC/Lead | JIRA Tickets | Importance/Usefullness (low/medium/high) |
---|---|---|---|---|
Schema Evolution | ability to gracefully evolve schemas | |||
Streaming | Streaming Dissemination - quotas, rate limiting, geofencing, filtering | |||
SQL | Support for SQL queries | |||
NanoCubes | ||||
Native API | finishing it, almost done. merge with blob store? no schema evolution concerns
| |||
Blob Store | More file formats (S3, other image formats, etc) | |||
Avro/JSON store | Don't transform to SFs, store native avro/json as "blobs" and index them
| |||
Vector Tiles |
| |||
Binary File Datastores | ||||
Count Min Sketch |
| |||
Attribute Bloom filters? |
|