Principal Data Engineer
Do you want to help eliminate barriers between ideas and business outcomes? We want you to bring your unique experiences and creative ideas to the table. CA Technologies provides software and solutions that help our customers to develop, manage, and secure complex IT environments to increase productivity and enhance competitiveness in their businesses. It's our aim to encourage global collaboration and results-oriented innovation, while supporting and developing our talented people and our communities. CA Technologies will empower you to drive authentic success, for both the business and yourself in the application economy.
The Principal Big Data Engineer works within the Enterprise Information Management team and is responsible for the ongoing development of the Enterprise Data Platform, critical to providing the bridge between actionable business insights and raw data. The Principle Big Data Engineer will be supporting a broad range of data pipelining needs accross all facets of the business, working with both internal and external data sources as needed.
This role will participate in all aspects of the data application life cycle, including analysis, design, development, testing, production deployment and support. The role will help formulate approaches to improve legacy process, develop opportunities and present innovative solutions on cutting-edge technologies. You will create big data accelerators to help deploy scalable solutions fast.
The Principle Big Data Engineer will work with Data Architects and Data Scientists to evolve the Enteprise Data platform, while delivering both strategic and tactical projects to provide iterative value through impactful business outcomes. This individual should be highly self-motivated with a strong problem-solving and analytical nature, love of learning, passion for insight, and desire for achieving results.
About CA:
CA Technologies is a Fortune 1000 company with a startup mentality - and we're searching for incredible, bright talent to dominate in the marketplace. Sure, CA has been a leading software company for nearly four decades, with a global customer base that includes the majority of the Fortune 2000 - but what excites us today is the opportunity to redefine the future of our industry in the age of the cloud, mobile, social and big data. We have a daring vision and a powerful, expanding solution set that helps the world's most successful companies realize their boldest objectives. For more information, visit .
Duties and responsibilities
Design and build data processing pipelines for structured and unstructured data using tools and frameworks in the big data ecosystem Architect, write code, complete programming and perform testing and debugging of data applications Design and develop new systems and tools in a number of languages to facilitate effective data ingestion and curation; batch, scalable event frameworks, streaming and real-time data analytic pipelines Build data APIs and data delivery services that support critical operational and analytical applications for our internal business operations, customers and partners. Implement and configure big data technologies as well as tune processes for performance at scale Design, develop and maintain conceptual, logical and physical data models for big data systems through schema on write and schema on read delivery. Perform data analysis with business, understanding business process and structuring both relational and distributed data sets. Build and provision analytical sandboxes for data scientists and analysts. Work with data science team to bring machine learning models into production Conduct timely and effective research in response to specific requests (e.g. data collection, summarization, analysis, and synthesis of relevant data and information) Devops for unit, functional, integration and regression test plans. Communicate with QA and port data engineering test scripts to QA team Evaluate, benchmark and integrate cutting edge open source data tools and technologies Monitor performance of the data platform and optimize as needed
Qualifications
Proficient understanding of distributed computing principles Programming development on Hadoop, include understanding of Zookeeper, Oozie and Yarn Working knowledge of HDFS storage formats (such as Parquet, ORC and ORC with snappy compression) Familiarity with networking and network application programming, including HTTP/HTTPS, JSON, and REST APIs Experience with at least one scripting language (ex:
Python) and one object oriented language (ex:
Java) Experience building ETL flows in Spark, Pyspark Experience in a production setting is preferred Knowledge of SQL-on-hadoop engines (presto, polybase, phoenix, etc) Ability to solve complex problems in a fast-paced environment with limited guidance An eye for quality and a willingness to do what is necessary to achieve deadlines in a dynamic environment with frequent priority changes Able to work efficiently in teams and/or as an individual Top-notch oral and written communication skills, especially the ability to make complex data and data science techniques accessible to non-technical team members B.S./M.S. in Computer Science or a related field, or equivalent experience
Desired Skills:
Unix/Linux OS level knowledge with Bash/Shell scripting is a plus Data Analysis:
Understand business processes, logical data models and relational database implementations Expert knowledge in SQL. Optimize complex queries. Experience working with cloud-based marketing and sales data a plus (Marketo, Salesforce, etc) Administration of Hadoop cluster, with all included services and ability to solve any performance and work load management issues on the cluster Experience with writing Kafka producers and consumers is a plus Experience with Spark Streaming, Spark SQL in a production setting is a plus
More About Working at CA:
CA has earned scores of global Workplace Excellence awards in the last few years - and there's a reason for that. Here you'll have the opportunity to explore flexible work arrangements, partner with our impressive customer set, and enjoy a competitive compensation package -all while pushing the boundaries of what's possible by collaborating with a diverse team of global innovators. In short? CA's fun, diverse, fast-paced culture have put us on the map as one of the best employers in IT. For more information, visit .
A Great CA Employee:
Takes smart risks Exhibits courage Is a driver Navigates ambiguity Embraces diversity Communicates openly Doesn't take themselves too seriously Likes to laugh Loves a good challenge Is resilient when faced with setbacks Teams well with others Insists on quality results Is forward-thinking Adapts easily to change Solves problems creatively
If you want to fulfill your potential, be acknowledged for your achievements, and be given autonomy to make decisions for your business and customers; if you want to work with a company that respects you as an individual - recognizing both your needs at work and your responsibilities outside of it - then CA Technologies is where you belong. At CA Technologies your passion and expertise can directly impact the business and you'll help offer our customers practical approaches to delivering new, innovative services and value through IT.
We offer competitive salary, company-sponsored premium Medical/Prescription & Dental Plans, company-paid Holidays, Vacation, Anniversary Service and Sick Days, 401(k) Plan, Education/Training Reimbursement, Charitable Gift Program, Adoption Assistance Program.
Our Islandia, NY campus offers several amenities, including:
Gourmet cafeteria with wide selection of cuisine
Montessori Child Development Center for children up to the age of six
Banking services
Fitness Center (state of the art facility complete with indoor full court basketball, indoor racquetball courts, cardio studio, outside running track, soccer fields, tennis courts)
Concierge services (dry cleaning, package shipping, stamps and movie rentals)
On-site chiropractor
At CA Technologies your passion and expertise can directly impact the business and you'll help offer our customers practical approaches to delivering new, innovative services and value through IT.
To learn more about CA Technologies and this opportunity, we welcome you to visit our web site at
We and all of our subsidiaries are equal opportunity employers. As such, it is our corporate policy to fill positions with qualified candidates regardless of the candidate's race, color, sex, age, religion, ancestry, national origin, citizenship status, marital status, sexual orientation, gender identity, genetic information, disability, pregnancy, military status, veteran status or any other protected group status.
Note to Recruiters and Placement Agencies:
We do not accept unsolicited agency resumes. Please do not forward unsolicited agency resumes to our website or to any of our employees. We will not pay fees to any third party agency or firm and will not be responsible for any agency fees associated with unsolicited resumes. Unsolicited resumes received will be considered our property and will be processed accordingly.
If you require an accommodation with the online application process, please contact Talent Acquisition at 1-.
EOE/Min/Women/Veterans/Disabled
Salary:
. Date posted:
28/04/2017Estimated Salary: $20 to $28 per hour based on qualifications.
|
|
 | |
