One stop solution to your remote job hunt!

By signing up you get access to highly customizable remote jobs newsletter, An app which helps you in your job hunt by providing you all the necessary tools.

OR
Subscribe to our highly customizable newsletter to get remote jobs from top remote job boards delivered to your inbox.
BambooHR about 2 years ago
3daccountingadwordsanalyticscloudcodeconsultcontentdirectoreduengineerengineeringexecutivefinancefinancialfront-endfull-timegogrowthhadoophealthillustratorinvestmentjuniorleadleadermanagementmanagermarketingmobileoperationalquality assurancerecruitmentsaassalesscrumsecurityseniorsoftwarestrategysystemtesttestingtrainingweb
Apply Now

What You'll DoAs a Senior Data Engineer on the data platform team, we’ll rely on your expertise across multiple disciplines to develop, deploy and support data systems, data pipelines, data lakes, and lakehouses.  Your ability to automate, performance tune, and scale the data platform will be key to your success.Your initial areas of focus will include:Collaborate with stakeholders to make effective use of core data assetsWith Spark and Pyspark libraries, load both streaming and batched dataEngineer lakehouse models to support defined data patterns and use casesLeverage a combination of tools, engines, libraries, and code to build scalable data pipelinesWork within an IT managed AWS account and VPC to stand up and maintain data platform development, staging, and production environmentsDocumentation of data pipelines, cloud infrastructure, and standard operating proceduresExpress data platform cloud infrastructure, services, and configuration as codeAutomate load, scaling, and performance testing of data platform pipelines and infrastructureMonitor, operate, and optimize data pipelines and distributed applicationsHelp ensure appropriate data privacy and securityAutomate continuous upgrades and testing of data platform infrastructure and servicesBuild data pipeline unit, integration, quality, and performance testsParticipate in peer code reviews, code approvals, and pull requestsIdentify, recommend, and implement opportunities for improvement in efficiency, resilience, scale, security, and performance What You Need to Get the Job Done (if you don’t have all, apply anyway!)Experience developing, scaling, and tuning data pipelines in Spark with PySparkUnderstanding of data lake, lakehouse, and data warehouse systems, and related technologiesKnowledge and understanding of data formats, data patterns, models, and methodologiesExperience storing data objects in hadoop or hadoop like environments such as S3Demonstrated ability to deploy, configure, secure, performance tune, and scale EMR and Spark Experience working with streaming technologies such as Kafka and KinesisExperience with the administration, configuration, performance tuning, and security of database engines like Snowflake, Databricks, Redshift, Vertica, or GreenplumAbility to work with cloud infrastructure including resource scaling, S3, RDS, IAM, security groups, AMIs, cloudwatch, cloudtrail, and secrets managerUnderstanding of security around cloud infrastructure and data systemsGit-based team coding workflows Bonus Skills (Not Required, So Apply Anyway!)Experience deploying and implementing lakehouse technologies such as Hudi, Iceberg, and DeltaExperience with Flink, Presto, Dremio, Databricks, or KubernetesExperience with expressing infrastructure as code leveraging tools like TerraformExperience and understanding of a zero trust security frameworkExperience developing CI/CD pipelines for automated testing and code deploymentExperience with QA and test automationExposure to visualization tools like Tableau Beyond the technical skills, we’re looking for iniduals who are:Clear communicators with team members and stakeholdersAnalytical and perceptive of patternsCreative in codingDetail-oriented and persistentProductive in a dynamic settingIf you love to learn, you’ll be in good company. You’ll likely have a Bachelor’s degree in computer science, information systems, or equivalent working experience. #Salary and compensation

No salary data published by company so we estimated salary based on similar jobs related to Education, Sales, Accounting, Finance, Senior, Design, Scrum, Mobile, Junior, Engineer, Illustrator, Digital Nomad, SaaS, Marketing, Testing and Cloud jobs that are similar:
$62,500 — $120,000/year#LocationCharleston, South Carolina, United States