Skip to main content

Replacing text and remove duplicates by first column

When working with plain text files like csv's you can use the following sentence to remove duplicates. It is a combination of sed and awk commands


 sed -E 's/([-]{1})([0-9]+)([-]{1})/-WER-/g' file.csv | awk -F- '!seen[$1]++' > outputfile.csv


Labels:

Comments

Post a Comment

Popular posts from this blog

Quarkus goes Reactive☢

In this tutorial I will explain how to create a quarkus based java application that will use reactive datasource to connect to a postgres database and be able to perform crud operations via rest endpoint. To make it more real this application will also be published into kubernetes cluster and will use configmap to store datasource configuration. To lern more about benefits of reactive data sources vs. jdbc regular data sources you can go to following link https://cutt.ly/1l260tK.  Step 1: Configure Project 
Post a Comment
Read more

Configuring web application with gwt+spring+hibernate

Building a web application using hibernate, spring and gwt. This time we will show how to create and cofigure an application that will use hibernate , spring and gwt. I will use HR sample database that comes with OracleXE. First , you have to create a GWT project (you can follow the steps for that on  https://developers.google.com/web-toolkit/ ) After you have the project created you need to create a tree of packages like that: Open HumanResources.gwt.xml and add the following line : By doing that you tell gwt to include model package and all dependent packages into gwt compilation Now we can start configuring spring and hibernate. to configure spring first you have to add the following lines on web.xml This will allow spring context to start doing its business. The you will have to create a field called "app-config.xml" in which you will define the settings for spring and hibernate. Over this app-config file first configure the properties as follo...
Post a Comment
Read more

Configuring web application with gwt+spring+hibernate (continued)

Last Time we learned how to configure a hibernate+ spring+ GWT application. Now we will continue explaining how I built this  application. First, we'll start with the Beans. For this example we will define one bean called Regions and an interface called Bean . Region will be the pojo class that contains the mapping for HR schema regions table The  @Entity  annotation is used to mark this class as an Entity bean. To use this annotation the class must have at least a package scope no-argument constructor. The  @Table  annotation is used to specify the table to persist the data. The  name  attribute refers to the table name. If  @Table  annotation is not specified then Hibernate will by default use the class name as the table name. The  @Id  annotation is used to specify the identifier property of the entity bean. The placement of the @Id  annotation determines the default access strategy that Hibernate will use f...
Post a Comment
Read more