* Fix _latest_partition_from_df in HiveEngineSpec
* Add test HiveEngineSpec._latest_partition_from_df
* Fix formatting to pass black
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/security/decorators.py", line 109, in wraps
return f(self, *args, **kwargs)
File "/app/superset/utils/log.py", line 164, in wrapper
value = f(*args, **kwargs)
File "/app/superset/views/core.py", line 2037, in extra_table_metadata
mydb, table_name, parsed_schema
File "/app/superset/db_engine_specs/presto.py", line 734, in extra_table_metadata
table_name, schema_name, database, show_first=True
File "/usr/local/lib/python3.7/site-packages/flask_caching/__init__.py", line 907, in decorated_function
rv = f(*args, **kwargs)
File "/app/superset/db_engine_specs/presto.py", line 977, in latest_partition
return column_names, cls._latest_partition_from_df(df)
File "/app/superset/db_engine_specs/hive.py", line 435, in _latest_partition_from_df
return [df.ix[:, 0].max().split("=")[1]]
File "/usr/local/lib/python3.7/site-packages/pandas/core/generic.py", line 5139, in __getattr__
return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'ix'
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
* fix(hive): Use parquet rather than textfile when uploading CSV files
* [csv/excel]: Use stream rather than temporary file
Co-authored-by: John Bodley <john.bodley@airbnb.com>
* changes to support presto impersionation with ldap
* renamed method to match 30 char limit
* import spell check
* added presto impersonation test
* refactored impersionation code to generalize for extension
* moving config_args mutation to the update_connect_args_for_impersonation
* moving config_args mutation to the update_connect_args_for_impersonation
* nits
* refactored update_impersonation_config method name to match lint rule
* reduced comment line length
* black reformats
Co-authored-by: rijojoseph01 <rijo.joseph@myntra.com>
* Support SET & SHOW commands as read only SQL commands
* Move is_readonly definition into the engine spec
* Rename & use super()
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
* feat: rename TABLE_NAMES_CACHE_CONFIG to DATA_CACHE_CONFIG
The corresponding cache will now also cache the query results.
* Slice use DATA_CACHE_CONFIG CACHE_DEFAULT_TIMEOUT
* Add test for default cache timeout
* rename FAR_FUTURE to ONE_YEAR_IN_SECS
* Athena.py: cretaed variable for too long line, removed disabled pylint rule.
* Enabled `line-too-long` in `drill.py`. Added variable for too long string.
* Reformatted hana.py:
- changed return statement into two lines to keep line lenght limit
* Enabling pylint rule in hive.py:
- `no-name-in-module` for pyhive in several places
- `line-too-long` and adding new lines
- `unused-import` is not invalid anymore
* Enabled `line-too-long` in `kylin.py`. Added variable for too long string.
* Enabled `unused_import` in `base.py` from db_engine_specs module.
* Enabled `unused_import` in `bigquery.py` from db_engine_specs module.
* Enabled `unused_import` in `druid.py` from db_engine_specs module.
* fix: athena datetime string converting
* chore: Adding pip-compile-multi et al
* Specify requirements.txt path for fossa
* [ci] Fixing CI
Co-authored-by: John Bodley <john.bodley@airbnb.com>
Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
* Support more table properties for the hive upload
Refactor
Add tests, and refactor them to be pytest friendly
Use lowercase table names
Ignore isort
* Use sql params
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
* fix: make auth for hive optional
This will make allow the admin to setup a root connection which can be impersonated.
* style: black lint conform
* style: clean up unnecessary condition expansion
* feat: Add new timegrains and convert_dttm to Druid engine spec
* Add TemporalType enum and fix test case
* Remove DATETIME for athena (original spec)
* Bump pylint version to 2.5.3
* Add a global disable for the most common new pylint error
* Fix a bunch of files containing very few errors
* More pylint tweakage, low-hanging fruit
* More easy stuff...
* Fix more erroring files
* Fix the last couple of errors, clean pylint!
* Black
* Fix mypy issue in connectors/druid/models.py
* [fix] csv upload when table metadata present
* Remove table from hive spec
* Move upload before table metadata creation
* Refine upload logic, dd unit tests and fix translations
* Use ALLOWED_EXTENSIONS from config
* Address review comments
* Fix error message grammar
* Add return type to hive csv upload and replace first with one_or_none
* Typo fix in CONTRIBUTING.md
* Alter references to config.get('FOO') to use preferred config['FOO']
* Set missing configuration constants in superset/config.py
* Misc. CI fixes
* Add type annotation for FEATURE_FLATGS
