Pandas Add Row to DataFrame – Definitive Guide
Pandas dataframe is a two-dimensional data structure. When using the dataframe for data analysis, you may need to create a new dataframe and selectively add rows for creating a dataframe with specific records.
You can add rows to the pandas dataframe using df.iLOC[i] = [‘col-1-value’, ‘col-2-value‘, ‘ col-3-value ‘] statement.
Other options available to add rows to the dataframe are,
- append()
- concat()
- iloc[]
- loc[]
If You’re in Hurry…
You can use the below code snippet to add rows to the dataframe.
Snippet
Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | India | Vikram | Aruchamy |
If You Want to Understand Details, Read on…
In this tutorial, you’ll learn the different methods available to add rows to a dataframe. You’ll also learn how to insert a row into an empty dataframe.
Table of Contents
Creating an Empty Dataframe
First, you need to create an empty dataframe to add rows to it. You can do it by using DataFrame() method as shown below.
Snippet
Empty dataframe is created as df .
Add Row to Dataframe
You can add rows to the dataframe using four methods. append() , concat() , iloc[] and loc[] .
Let’s have a look at it one by one.
To create a new row, you need to know the columns already available in the dataframe. Read How to Get Column Name in Pandas to know the columns in the dataframe.
Alternatively, you can print the dataframe using print(df) to know the dataframe columns.
Using Append
You can use the append() method to append a row to an existing dataframe.
Parameters
- dictionary or Pandas Series or Dataframe – Object with values for new row
- ignore_index = True Means the index from the series or the source dataframe will be ignored. The index available in the target dataframe will be used, instead. False means otherwise. This is optional. Returns
- A resultant dataframe which has the rows from the target dataframe and a new row appended.
inplace append is not possible. Hence, do not forget to assign the result to a dataframe object to access it later.
In the below example, a dictionary is created with values for the columns which already exist in the target dataframe. Then it is appended to the target dataframe using the append() method.
Now, you’ve appended one row to the dataframe.
Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | India | Vikram | Aruchamy |
This is how you can insert a row to the dataframe using append.
Using Concat
You can append a row to the dataframe using concat() method. It concatenates two dataframe into one.
To add one row, create a dataframe with one row and concatenate it to the existing dataframe.
Parameters
- List of dataframes – List of dataframes that needs to be concatenated
- ignore_index – Whether the index of the new dataframe should be ignored when concatenating to the target dataframe
- axis = 0 – To denote that rows of the dataframe needs to be converted. If you want to concatenate columns, you can use axis=1 Returns
It returns a new dataframe object which has the rows concatenated from two dataframes.
inplace concatenation is not supported. Hence, remember to assign the result to a variable for later use.
Snippet
In the above example, you’re creating a new dataframe with one row and it is named as df2 . You’re concatenating this to dataframe df which already has one dataframe in it.
Both df and df2 will be concatenated and you’ll see two rows in the resultant dataframe.
Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | India | Vikram | Aruchamy |
| 1 | India | Kumar | Ram |
This is how you can use the concat() method to add rows to the dataframe.
Using iLOC
You can use the iLoc[] attribute to add a row at a specific position in the dataframe. iloc is an integer-based indexing for selecting rows from the dataframe. You can also use it to assign new rows at that position.
Adding a row at a specific index position will replace the existing row at that position.
When you’re using iLoc to add a row, the dataframe must already have a row in the position. At least an empty row. If a row is not available, you’ll see an error IndexError: iloc cannot enlarge its target object . iLoc will not expand the size of the dataframe automatically.
Snippet
In the above example, you’re directly adding a row at the index position 1 . It replaced the values available in that position with the new values.
Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | India | Vikram | Aruchamy |
| 1 | India | Shivam | Pandey |
This is how you can use the iloc[] to insert a row to the existing dataframe.
Using LOC
You can add a row to the dataframe using the loc parameter. loc[] is used to access a set of rows from the dataframe using the index label. You can also assign rows with a specific index label using the loc attribute.
When using the loc[] attribute, it’s not mandatory that a row already exists with a specific label. It’ll automatically extend the dataframe and add a row with that label, unlike the iloc[] method.
A full program is demonstrated for this method because previous methods have the dataframe with the row indexes 1,2,3.
To demonstrate loc using the row indexes with names like a , b , a new dataframe is created with labels a and b . Then a new row is assigned with the row label c using the loc[] method.
Snippet
First a dataframe df3 is created with two rows with label a and b . Then a row is inserted with the label c using the loc[] method.
Dataframe Will Look Like
This is how you can use the loc[] method to add rows to the dataframe. Either it is an empty dataframe or it already has values.
Once the rows are added, you select rows from pandas dataframe based on column values to check if the rows are added properly.
Next, you’ll see the different circumstances where you can use the loc , iloc , append() or concat() methods to add rows to the dataframe.
Pandas Insert Row at Specific Index
You can insert rows at a specific index in a dataframe using the loc method.
This will be useful when you want to insert a row between two rows in a dataframe.
Alternatively, you can also use the iloc[] method to add rows at a specific index. However, there must be a row already existing with a specific index.
Note
When using loc[] , If a row is already existing with that index label, it’ll be replaced with the new row.
Snippet
A row will be added with the index label 2 .
Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | India | Vikram | Aruchamy |
| 1 | India | Shivam | Pandey |
| 2 | India | Shivam | Pandey |
This is how you can append rows at a specific index in a dataframe.
Pandas Insert Row At top
You can insert a row at the top of the dataframe using the df.loc[-1] .
After inserting the row with index -1 , you can increment all the indexes by 1 .
Now indexes of the rows in the dataframe will be 0,1,2. n-1.
Note
To use this method, the index labels of the rows must be integers. Otherwise, it won’t work.
Snippet
A row is first added at position -1 and then all the indexes will be incremented and sorted.
Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | India | Raj | Kumar |
| 1 | India | Vikram | Aruchamy |
| 2 | India | Shivam | Pandey |
| 3 | India | Shivam | Pandey |
This is how you can insert a row at top of the dataframe.
Pandas Insert Row at Bottom
You can insert a row at the bottom in the dataframe using the df.loc[df.shape[0]] .
df.shape[0] returns the length of the dataframe.
For example, if a dataframe already contains 3 rows, already existing rows will have the index 0,1,2,3. Shape[] method will return 4 . Hence when you insert using loc[4] , a row will be added at bottom of the dataframe which has the index 4 .
Snippet
A new row will be added at the index position 4 as you see below.
Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | India | Raj | Kumar |
| 1 | India | Vikram | Aruchamy |
| 2 | India | Shivam | Pandey |
| 3 | India | Shivam | Pandey |
| 4 | India | Krishna | Kumar |
This is how you can append a row at the bottom of the dataframe using loc[] .
Pandas Insert Empty Row
You may need to append an empty row to the pandas dataframe for adding a row to it later. You can also fill values for specific columns in the dataframe after creating an empty row.
Empty rows can be appended by using the df.loc[df.shape[0]] and assigning None values for all the existing columns.
For example, if your dataframe has three columns, you can create a series with 3 None values and assign it at the last position of the dataframe.
That is how you can insert an empty row into the dataframe.
Snippet
An empty row is added at the end of the dataframe.
Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | India | Raj | Kumar |
| 1 | India | Vikram | Aruchamy |
| 2 | India | Shivam | Pandey |
| 3 | India | Shivam | Pandey |
| 4 | India | Krishna | Kumar |
| 5 | None | None | None |
This is how you can add an empty row to the end of the dataframe.
Pandas Append Two Dataframe Pandas
You can append a dataframe to another dataframe using the dataframe append() method.
append() method accepts a dataframe and appends it to the calling dataframe and returns a new dataframe object.
inplace append is not possible. hence you need to assign the result a dataframe object if you want to use it later.
ignore_index can be used to ignore the index of the dataframe that is assigned to the target dataframe.
Snippet
In the above example, dataframe df2 is appended to df and assigned it back to the df object.
Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | India | Raj | Kumar |
| 1 | India | Vikram | Aruchamy |
| 2 | India | Shivam | Pandey |
| 3 | India | Shivam | Pandey |
| 4 | India | Krishna | Kumar |
| 5 | None | None | None |
| 6 | India | Vikram | Aruchamy |
This is how you can append two dataframe in pandas using the append() method.
Why You Should Not Add Rows One By One To Dataframe
You may need to create a dataframe and append one row at a time in various scenarios.
In that case, it is advisable to create a list first to hold all the records and create a dataframe with all the records in one shot using the pd.DataFrame() method.
Calling the append() method for each row is a costlier operation. But adding the rows to the list is not costlier. Hence, you can add to the list and create a dataframe using that list.
Snippet
For more details about this scenario, refer StackOverflow answer.
Dataframe Will Look Like
| First Name | Last Name | Country | |
|---|---|---|---|
| 0 | Krishna | Kumar | India |
| 1 | Ram | Kumar | India |
| 2 | Shivam | Pandey | India |
This is how you can create a pandas dataframe by appending one row at a time.
Conclusion
To summarize, you’ve learned how to create empty dataframe in pandas and add rows to it using the append() , iloc[] , loc[] , concatenating two dataframes using concat() .
Also, how these methods can be used to insert a row at a specific index, add a row to the top or bottom of the dataframe, how to add an empty row to the dataframe which can be used at a later point.
In addition to that, you’ve learned why you should not create a pandas dataframe by appending one row at a time and use a list in such scenarios and create a dataframe using the list.
Add Row to Dataframe in Pandas
Pandas Dataframe provides a function dataframe.append() to add rows to a dataframe i.e.
Here, the ‘other’ parameter can be a DataFrame or Series or Dictionary or list of these. Also, if ignore_index is True then it will not use indexes.
Examples of adding row to the dataframe
Suppose we have a dataframe df, whose contents are as follows,
Add dictionary as a row to dataframe
In dataframe.append() we can pass a dictionary of key value pairs i.e.
- key = Column name
- Value = Value at that column in new row
Let’s add a new row in above dataframe by passing dictionary i.e.
It will not modify the existing dataframe object mod_df, it will return a new dataframe containing copy of contents of existing dataframe and with a new row appended at it’s end. Contents of the dataframe returned are,
New DataFrame’s index is not same as original dataframe because ignore_index is passed as True in append() function. Also, for columns which were not present in the dictionary NaN value is added.
Passing ignore_index=True is necessary while passing dictionary or series otherwise following TypeError error will come i.e.
“TypeError: Can only append a Series if ignore_index=True or if the Series has a name”
Complete example to add a dictionary as row to the dataframe is as follows,
Output:
Add Series as a row in the dataframe
We can also pass a series object to the append() function to append a new row to the dataframe i.e.
While creating a series object we passed the index names same as index of dataframe. Contents of the dataframe returned are,
Checkout the complete example to a append a series as row to dataframe,
Output:
Add multiple rows to pandas dataframe
We can pass a list of series too in the dataframe.append() for appending multiple rows in dataframe. For example, we can create a list of series with same column names as dataframe i.e.
Now pass this list of series to the append() function i.e.
Contents of the dataframe returned are,
Complete example to add multiple rows to dataframe is as follows,
Output
Add row from one dataframe to another dataframe
We can select a row from dataframe by its name using loc[] attribute and the pass the selected row as an argument to the append() function. It will add the that row to the another dataframe. Let’s see an example where we will select a row with index label ‘b’ and append it to another dataframe using append(). For example,
Output
Add list as a row to pandas dataframe using loc[]
Adding a list as a row to the dataframe in pandas is very simple and easy. We can just pass the new index label in loc[] attribute and assign list object to it. For example,
It will append a new row to the dataframe with index label ‘k’. Let’s see a complete example to append a list as row to the dataframe,
Output:
Add a row in the dataframe at index position using iloc[]
We can add a row at specific position too in the dataframe using iloc[] attribute. Checkout the example, where we will add a list as the 3rd row the dataframe. For example,
Output:
Summary:
We learned about different ways to add / append rows to the dataframe in pandas.
Pandas Tutorials -Learn Data Analysis with Python
Pandas Tutorial Part #1 — Introduction to Data Analysis with Python
Pandas Tutorial Part #2 — Basics of Pandas Series
Pandas Tutorial Part #3 — Get & Set Series values
Pandas Tutorial Part #4 — Attributes & methods of Pandas Series
Pandas Tutorial Part #5 — Add or Remove Pandas Series elements
Pandas Tutorial Part #6 — Introduction to DataFrame
Pandas Tutorial Part #7 — DataFrame.loc[] — Select Rows / Columns by Indexing
Pandas Tutorial Part #8 — DataFrame.iloc[] — Select Rows / Columns by Label Names
Pandas Tutorial Part #9 — Filter DataFrame Rows
Pandas Tutorial Part #10 — Add/Remove DataFrame Rows & Columns
Pandas Tutorial Part #11 — DataFrame attributes & methods
Pandas Tutorial Part #12 — Handling Missing Data or NaN values
Pandas Tutorial Part #13 — Iterate over Rows & Columns of DataFrame
Pandas Tutorial Part #14 — Sorting DataFrame by Rows or Columns
Pandas Tutorial Part #15 — Merging or Concatenating DataFrames
Pandas Tutorial Part #16 — DataFrame GroupBy explained with examples
Are you looking to make a career in Data Science with Python?
Data Science is the future, and the future is here now. Data Scientists are now the most sought-after professionals today. To become a good Data Scientist or to make a career switch in Data Science one must possess the right skill set. We have curated a list of Best Professional Certificate in Data Science with Python. These courses will teach you the programming tools for Data Science like Pandas, NumPy, Matplotlib, Seaborn and how to use these libraries to implement Machine learning models.
Checkout the Detailed Review of Best Professional Certificate in Data Science with Python.
Remember, Data Science requires a lot of patience, persistence, and practice. So, start learning today.
pandas.DataFrame.append¶
Append rows of other to the end of caller, returning a new object.
Deprecated since version 1.4.0: Use concat() instead. For further details see Deprecated DataFrame.append and Series.append
Columns in other that are not in the caller are added as new columns.
Parameters other DataFrame or Series/dict-like object, or list of these
The data to append.
ignore_index bool, default False
If True, the resulting axis will be labeled 0, 1, …, n — 1.
verify_integrity bool, default False
If True, raise ValueError on creating index with duplicates.
sort bool, default False
Sort columns if the columns of self and other are not aligned.
Changed in version 1.0.0: Changed to not sort by default.
A new DataFrame consisting of the rows of caller and the rows of other .
General function to concatenate DataFrame or Series objects.
If a list of dict/series is passed and the keys are all contained in the DataFrame’s index, the order of the columns in the resulting DataFrame will be unchanged.
Iteratively appending rows to a DataFrame can be more computationally intensive than a single concatenate. A better solution is to append those rows to a list and then concatenate the list with the original DataFrame all at once.
With ignore_index set to True:
The following, while not recommended methods for generating DataFrames, show two ways to generate a DataFrame from multiple data sources.