Transformation in Talend ETL by using tMap and condition-based filter

| January 8, 2019

article image
This blog will be focused on Transformation that how to create a job that uses tMap component by adding condition-based filters. For this purpose, we have a scenario like a data file named movies. BI Company in Saudi Arabia offering exclusive services by working on this we need to Transform (filter) movies according to their release years, here we need to create three files one who contains these movies which was released in 90s, and second which was released in 80s and the remaining are all other movies categories Like below e.g. For this purpose, there are only few steps which we need to create this job. Those steps are following below: Create a job and add tMap component. Add a simple filter. Store the result in a simple file. Create a second filter. Create a filter reject output. We need to create a new job and adding my movies metadata file as a tFileInputDelimited component and to create new filter we also need to add tMap  and SAP BO Business Objects services in Saudi Arabia component in a job to create the flow of data which we need to connect with both components and give new name(movies).

Spotlight

ClearObject

ClearObject is an IoT systems integrator that designs, develops, deploys and manages innovative connected solutions and data-driven digital products.

OTHER ARTICLES

Time Machine Big Data of the Past for the Future of Europe

Article | February 24, 2020

Emerging technology has the power to transform history and cultural heritage into a living resource. The Time Machine project will digitise archives from museums and libraries, using Artificial Intelligence and Big Data mining, to offer richer interpretations of our past. An inclusive European identity benefits from a deep engagement with the region’s past. The Time Machine project set out to offer this by exploiting already freely accessible Big Data sources. EU support for a preparatory action enabled the development of a decade-long roadmap for the large-scale digitisation of kilometres of archives, from large museum and library collections, into a distributed information system. Artificial Intelligence (AI) will play a key role at each step, from digitisation planning to document interpretation and fact-checking. Once embedded, this infrastructure could create new business and employment opportunities across a range of sectors including ICT, the creative industries and tourism.

Read More

Living upto Learn, Re Learn and Unlearn

Article | March 23, 2021

Learn, re Learn and Unlearn The times we are living in, we have to upgrade ourselves constantly in order to stay afloat with the industry be it Logistics, Traditional business, Agriculture, etc.. Technology is constantly changing our lives the way we used to live, living and will live. Anyone who thinks technology is not their cup of tea then I would say he /she will have no place in the world to live. It’s a blessing or curse on human race, only time will tell but effects are already surfacing in the market in the form of Job cut, poverty, some roles are no longer needed or replaced with. Poor is getting poorer and rich is getting richer. Covid19 has not only brought the curse on human race but it has been a blessing in disguise for Tech giants and E-commerce. Technology not only changing the business but every human’s outlook towards life, family structure, the globalization of talents etc. It is nerve wrenching to imagine just what the world will look like in coming 20 years from now. Can all of us adapt to learn, re learn and unlearn quote? Or we have to depend upon countries/Governments to announce Minimum Wage to sustain our basic needs? Uncertainties are looming as the world is coming closer due to technology but emotionally going far. It’s sad to see children, colleagues communicating via emails and messages in the same home and office. Human is losing its touch and feel. Repercussion to resists of learning, unlearning and relearning can bring down choices to none in the long run. Delay in adapting to change can be increasingly expensive as one can lose their place in a world earlier than one think. From 1992, where fewer people used to have facility of internet around , People used to stay in jobs for life but same people are now not wanted in the jobs when they go for interview as they lack in experience just because they have been doing what they were doing in one job without exposing themselves to the world’s new requirement of learn , re learn and unlearn. Chances of this group, getting a job will be negative. World has thrown different types of challenges to people, community, jobs, businesses , those people used to be applauded for remaining On one job for life ,same group of people are looked differently by corporate firms as redundant due to technology. So should people keep changing jobs after few years to just get on to learn, re learn and unlearn or continue waiting for their existing companies to face challenges and go off from the market? Only time and technology will determine what is store for human race next. According to some of the studies, its shown the longer the delay in adopting technology for any given nation, the lower the per capita income of that nation. It shows extreme reliance on Technology but can all of us adopt to the technology at the same rate as its been introduced to us? Can our children or upcoming next generations adopt technology at same scale? Or future is Either Technology or nothing, in Short Job or Jobless there is no in between option? Stephen Goldsmith, director of the Innovations in Government Program and Data-Smart City Solutions at the John F. Kennedy School of Government at Harvard University, said that in some areas, technological advancements have exceeded expectations made in 2000. The Internet also has exploded beyond expectations. From 2000 to 2010, the number of Internet users increased 500 percent, from 361 million worldwide to almost 2 billion. Now, close to 4 billion people throughout the world use the Internet. People go online for everything from buying groceries and clothes to finding a date. They can register their cars online, earn a college degree, shop for houses and apply for a mortgage but again same question is arising , Can each one of us at the same scale use or advance their skill to use technology or we are leaving our senior generations behind and making them cripple in today’s society? Or How about Mid age people who are in their 50s and soon going to take over senior society , Can they get the job and advance their skill to meet technology demands or learn, unlearn and re learn or Not only pandemic but even Technology is going to make human redundant before their actual retirement and their knowledge, skill obsolete. There should be a way forward to achieve balance, absolute reliance on Technology is not only cyber threat to governments but in long term, Unemployment, Creating Jobs or paying minimum wage to unemployed mass will be a huge worry. At the end of the day, humans need basic and then luxury. Technology can bring ease of doing business, connecting businesses and out flows, connecting Wholesalers to end users but in between many jobs, heads will be slashed down and impact will be dire. Therefore Humans have to get themselves prepared to learn, unlearn and re learn to meet today’s technology requirement or prepare themselves for early retirement.

Read More

CAN QUANTUM COMPUTING BE THE NEW BUZZWORD

Article | March 30, 2020

Quantum Mechanics created their chapter in the history of the early 20th Century. With its regular binary computing twin going out of style, quantum mechanics led quantum computing to be the new belle of the ball! While the memory used in a classical computer encodes binary ‘bits’ – one and zero, quantum computers use qubits (quantum bits). And Qubit is not confined to a two-state solution, but can also exist in superposition i.e., qubits can be employed at 0, 1 and both 1 and 0 at the same time.

Read More

Soft Skills in Data Science

Article | April 29, 2021

We live in a world convulsed by new technologies and we are witnessing how more and more processes are automated in order to be executed with the same skill or even with better results than if they were carried out by a human, all this in order to be more efficient and effective. In this context the world of work is becoming increasingly competitive, because to remain employable we need to learn to manage or find a way to adapt our knowledge and skills to new technologies. With the spread of e-learning platforms and the tutorials that we can find available on the internet, acquiring new knowledge is within everyone's reach. For this reason, it is necessary to differentiate ourselves in order to stand out from other professionals, who have the hard skills similar to ours and this is precisely where Soft Skills play a very important role. What are Soft Skills? Soft skills are actually a combination of individual social skills, communication skills, personality traits, attitudes, social intelligence and emotional intelligence. Which facilitate relationships with others, making us more effective when interacting with other people. We could say that Soft Skills are the human interface that allow us to adapt to different working environments and industries. They are powerful tools for personal and professional growth. Why are Soft Skills key in our professional growth? Nowadays, standing out in the world of work is getting increasingly difficult, regardless of whether you are part of a corporation or work independently, due to the great competition within the labor market. That is why we must develop certain skills and attitudes that help us to function properly and successfully meet professional demands. Soft Skills are the point of differentiation that allows us to be selected for a position. The reason is very simple, we could be applying for a position and competing with people that are equal or even more qualified than us at a technical level, but to achieve the collaborative objectives of the company, more is required than just the technical and rational part. Also the way of communicating, values, ethics, as well as personality traits are highly valued factors since they help to drive organizations through high-performance teams, guaranteeing the achievement of their objectives. The background of the Soft Skills that we have trained throughout our lives make us unique, because it is unlikely that two people have the same combination of Soft Skills and been trained in a similar way, and that makes us more competitive against certain job opportunities where perhaps many will have the same Hard Skills, but where our Soft Skills will be the ones that will make us stand out to continue advancing in our professional career. How to sharpen our Soft Skills? To perform in any job we necessarily need to interact with other people, even if we work independently or remotely, so we must have the necessary skills that allow us to connect successfully with our teammates and stakeholders. Starting from the fact that Soft Skills are human skills, we can say that we have them pre-installed and the way to start using them (installing them) is through the experiences we undergo every day. Imagine being able to communicate assertively in your work environment and in your personal life. Master the use of tools installed in you to improve your interpersonal relationships within your work teams and reduce conflict. This would allow you to foster a healthy working environment and be able to lead any team in any environment in a strategic and effective way. Think of Soft Skills as a set of Apps that are ready to be used (like a toolbox) and that according to the experiences that are presented in our personal and / or professional lives, we are going to choose to use these applications to achieve our goals. Every time we access one of these applications, we are giving it the opportunity to collect data that will allow it to personalize its insights according to our needs and to fine-tune its effectiveness each time we use it. One of the best ways to train our Soft Skills is by leaving our comfort zone, because that will allow us to 'install' more and more Soft Skills. Another way to refine our Soft Skills is by participating in activities that involve people we do not know and even better if we involve people from other cultures, because we will achieve a beneficial exchange of experiences and knowledge for both parties that will enrich and make the training of our Soft Skills even more valuable. Some examples of activities that will enhance your Soft Skills: • Participate in competitions (e.g. Hackathons) • Found or be a lead of a community that shares your interests, and organizes small or large projects. • Organize a study group aimed at carrying out a technical or business project in order to confront professionals from various fields or industries. • Find resources and experts to help you. There are Soft Skills trainers who know useful techniques and tips to develop/sharpen your skills. • Participate in volunteer activities. You will meet new people with whom to put your Soft Skills in action. These activities will train/sharpen your leadership skills, teamwork, delegation, interpersonal communication, persuasion, etc. These are skills that we do not have as much facility to train while we are students or when we have just started working after finishing our studies, and that are required in the labor market to continue climbing in our professional career. Why do Soft Skills matter in the Data Science universe? A consequence of the use of Artificial Intelligence and Data Science is that many of the jobs that we know today will be automated and this is a matter of concern for many professionals who see their careers are in danger, but the good news is that in the future many new jobs the Soft Skills will be the main protagonists, this is what John Thompson explains us in his book "Building Analytics Teams" In other words, it is precisely our human skills that will allow us to be more employable in the future, and they will be highly requested skills because according to what the experts envision which is, that the machines will not be able to match us in this field, and that is why training our Soft Skills becomes a priority because they will allow us to be the key players of the future. On the other hand, Data Science is an interdisciplinary field where Soft Skills such as cooperation and communication are essential to achieve the goals set. Denis Rothman, author of the book "Transformers for Natural Language Processing" in an interview that I conducted, mentioned that The Human Quality is the most important thing for him when choosing the members of his work team. These are some considerations to take into account to generate a culture of cooperation: • People work harder and need less supervision, when they themselves control their work and have more freedom to choose how to do it. When they work as a team, they show greater motivation, their sense of pride increases and productivity reaches higher levels. • Solid teams that seek quality and excellence correct themselves; that is, they identify problems and correct them very quickly. Thus, they gain work experience and increase their performance. • Forming a solid and efficient work team requires patience. You need to give them time to see your results. They will have to establish procedures to complete tasks, handle administrative functions and work together efficiently, they will even have to adapt to their own decisions and accept their consequences. • A manager or team leader must recognize the team building process without expecting immediate results. The group will have to go through a learning process and this will take longer in some groups than in others. Another key component to achieving high levels of cooperation is fluid communication among team members and stakeholders. For instance defining the communication channels and the contact points in the different teams involved, guarantees the constant flow of communication during the life cycle of a Data Science project. One of the most critical moments is the presentation of the results to the stakeholders. In some cases the results of a project are not taken into consideration not so much because the expected results are not achieved, but because the way in which these results are presented are not meaningful for the stakeholders, and this, in most cases, it is due to the existence of communication barriers that is a consequence of the use of a language (terminologies) used in the technical world but not in the business world. After taking a tour of the world of Soft Skills, we can conclude by saying that Soft Skills are like superpowers that are waiting for the opportunity to be put into action, to make you a superhero or superheroine. Keep climbing positions in your professional career depends on you, on how much you use these superpowers but above all on your skills to refine them and make them available to the work team of which you are part. Don't wait any longer and start discovering your potential, start training your Soft Skills! If you want to know more about Soft Skills, I invite you to visit The Soft Skills Show

Read More

Spotlight

ClearObject

ClearObject is an IoT systems integrator that designs, develops, deploys and manages innovative connected solutions and data-driven digital products.

Events