Post Your Answer

Easy and simple way for missing data estimation?

3 years ago in Data Analysis , Data Mining , Matlab , SPSS By Jeremy

Hi everyone, I know this might give a wrong idea but please read this description. I am not trying to save the efforts (that is not the case AT ALL). It’s just that I have to obtain climatological data for my research.The data has to be based on daily observation for about 20 years. So, as you can see now, I really have to know the method of estimating the missing climatological data.

Out of all the variables I included the rain estimates and sunshine hours are the ones which are creating issues in the data. The defect in these often diffuses the series by creating problems in data of a few months and sometimes even years.

I need to know of some reliable method in which I can settle this issue and estimate the missing data. Please help me if you have any previous experience in such research work or if you can share some similar research articles or documents.

I hope you help a fellow research student.

All Answers (4 Answers In All)

By Abeden Answered 3 years ago

Hello dear, I would suggest you use the LOESS (Locally Weighted Regression Smoother) method to estimate the missing data. It is an effective method and works really well. Also, in case you are well aware of the matlab process, then I would like you to go through the given page link once. Maybe it will be of some help to you. Data Driven Fitting with Matlab You can also read his article for a better understanding. Let me know if it helps. Comparison of different Methods for Univariate Time Series Imputation in R

By Manoj Answered 3 years ago

You can use matlab’s function as Adeben suggested. Make sure that your raw data is either in csv file or xlsx file before putting it on matlab. That way matlab will save it automatically in the matrix or cell format. After that you can use the find function to get the raw missing data or the logical judgment command (a=NaN) can also be used. With this you will be able to get the number of missing data quite easily.

Replied 3 years ago

By Jeremy

Hello Manoj, I am not having the issue with the count, I am having issue with the estimation of missing data. Thanks for your responses.

Replied 3 years ago

By Manoj

Hello Jeremy, I apologize for not understanding the question correctly. And well, in that case, you can use the EM algorithm or try using the mean/average to estimate the missing data. Through this you can select the multiple imputation approach maybe. Other than this, there is another way. If you already know the places where the data is missing, then use the estimating rule and use Matlab (or other software as per your choice) to calculate the data by using the loop command. I hope this helps.

Replied 3 years ago

By Jeremy

Thank you again, Manoj. I will try the methods but as for the multiple imputation, I don’t think that the data I have will be enough for it. But I will try these methods after studying them to see if they work with my research. Thank you so much.

By Siti Answered 3 years ago

No matter how I see it and how long the gap seems, using estimation and imputation is never the way of filling the gaps. Instead of using these dirty tricks to complete the ask quickly, I would suggest you to study the relative articles and research data from those years. The climatological imputation or else, it can never be close to truth/reality when it is done through imputation. Every case needs to be studied well, the days, months, and years are no joke. You should fill the data case by case, and for that compare the duration you included with other studies.

By Shubham Dhingra Answered 3 years ago

Hello Jeremy, there is a method of simply estimating the missing data. You can apply the moving average in which generally 3 previous missing points and 3 next daa points are estimated. This method works well but I am not too sure whether it works with data gaps that occur continuously and multiple times. But you can always try it. Also, you can use hydrolab. If you don’t know much about it, then take help from experts. I took it once and my research paper got accepted too.

View Related Questions