Job title: Data Engineer
Location: UK
Duration: 6-month contract Inside IR35
Working Mode: Fully Remote
Vetting: Active SC Clearance
Job Description:
Data Engineer Skills and Technologies Data Engineering:
– Strong understanding of data concepts
– data types, data structures, schemas (both JSON and Spark), schema management etc
– Strong understanding of complex JSON manipulation – Experience working with Data Pipelines using a custom Python/PySpark frameworks
– Strong understanding of the 4 core Data categories (Reference, Master, Transactional, Freeform) and the implications of each, particularly managing/handling Reference Data.
– Strong understanding of Data Security principles
– data owners, access controls
– row and column level, GDPR etc including experience of handling sensitive datasets
– Strong problem solving and analytical skills, particularly able to demonstrate these intuitively
Requirements
Required Skills:
Languages / Frameworks
– JSON
– YAML
– Python (as a programming language, not just able to write basic scripts. Pydantic experience would be a bonus.)
– SQL
– PySpark
– Delta Lake
– Bash (both CLI usage and scripting)
– Git
– Markdown
– Scala (bonus, not compulsory)
– Azure SQL Server as a HIVE Metastore (bonus) Technologies
– Azure Databricks
– Apache Spark
– Delta Tables
– Data processing with Python
– PowerBI (Integration / Data Ingestion)
– JIRA
...companys Cash Offer, Trade-In, Simple Sale, listing management, mortgage, and title/escrow products. Draft and negotiate contracts... ..., advertising, IP, and privacy laws and regulations to ensure compliance. in house jobs, go in house with jdhuntr in house jobs...
Associate Director Client Services & New Business Development At our Company, we grow People, Brands, and Businesses! We are seeking a highly dynamic Associate... ...role is to provide direction to the planning, management, and oversight of events and other special projects...
...all new and existing clients while supporting the overall sales and operations of the store. Who are We Hive & Colony combines fashion and technology and innovation to make premium, tailor-made suits, shirts and other tailored made garments for men and women. We...
...Estate Pros where your creative energy meets unlimited potential! We're on the hunt for new faces and fresh ideas people who can show up on time, follow through, and thrive in a dynamic, supportive environment. Whether you're into acting, photography, filmmaking,...
Title: Backlist Buyer (Religion)Category: Merchandising & Product DevelopmentEmploymentType: Full-TimeLocation: Remote JobLocationType: corporateJobLocation: ANYWHERE USA, New York 12345JobSummary: In this role, you bring expert book knowledge of subjects to support...