Skip to main content

Replacing text and remove duplicates by first column

When working with plain text files like csv's you can use the following sentence to remove duplicates. It is a combination of sed and awk commands


 sed -E 's/([-]{1})([0-9]+)([-]{1})/-WER-/g' file.csv | awk -F- '!seen[$1]++' > outputfile.csv


Comments

Popular posts from this blog

Quarkus goes Reactive☢

In this tutorial I will explain how to create a quarkus based java application that will use reactive datasource to connect to a postgres database and be able to perform crud operations via rest endpoint. To make it more real this application will also be published into kubernetes cluster and will use configmap to store datasource configuration. To lern more about benefits of reactive data sources vs. jdbc regular data sources you can go to following link https://cutt.ly/1l260tK.  Step 1: Configure Project 

Register new WCS store view by using database

To register a new view by using SQL queries: insert into acaction (acaction_id, action) values ((select counter from keys where tablename='acaction'), 'NewView');  insert into acactactgp (ACACTGRP_ID,ACACTION_ID) values  ((SELECT ACACTGRP_ID FROM ACACTGRP WHERE GROUPNAME = 'AllSiteUsersViews'  and member_id in (select orgentity_id from orgentity where orgentityname='Root Organization')  ),  (select acaction_id from acaction where action='NewView'));  UPDATE KEYS SET COUNTER = COUNTER+1 WHERE TABLENAME = 'acaction'; commit