Kiva update
Feb. 18th, 2009 12:28 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
gementShort form: If you are investing in microloans at http://www.kiva.org, diversify as much as you can among partner organizations, and do not rely on the fact that an organization has 4 or 5 stars. Kiva's statistics on their own ratings system are inaccurate and need to be tracked better.
Belated Edit: The tracking is not as poor as first implied. I got a later response clarifying this, which hpet has pasted in a comment at the bottom. Check it out. They still have said in other places that their star ratings are not very precise, though.
The statistics at http://www.kiva.org/about/risk/statistics/ say that no loan from a 4 or 5 star organization has defaulted ever. As in ever. I thought that was pretty amazing. It would be, except it's not true, it's poor data tracking on their part.
I did a whole pile of digging around in their default statistics (I'm not worried about delayed payments, just payments that don't come through in the final tally). My findings are laid out in the comments here: http://gement.livejournal.com/96419.html and most of those findings are valid. However, based on the input from a pissed-off investor in MIFEX loans, also in those comments, I recognized that Kiva might not be doing a good job of data tracking and analysis.
(So I sent them this question...)
Subject: Our Field Partners - When ratings change, does it get tracked in the statistics?
I'm trying to understand how changed ratings get tracked in the site statistics on defaults, etc. I'm tracking the MIFEX case, of course, and I see that between May and November you were offering loans through them with a 1-star rating.
When everything defaulted, did the loans made when they were rated 3 stars stay in the 3 star statistics and the loans made when they were rated 1 star go in the 1 star statistics?
I'm asking because someone told me that he invested in MIFEX when they were rated 4 stars and lost money. Since you do not have ratings history (please add this as soon as possible), I cannot verify whether they were ever rated 4 stars. If a loan was made when they were 4 stars, and it defaulted, would it have ended up in the 4 star statistics, or moved down with the partner's adjusted rating?
(Obviously, I think the loan should be counted in the statistics for the star rating it was offered under. I just want to find out if that's how it works _now_, and if not, how soon you plan to fix that.)
(salutation, end transmission. Their reply, within one day...)
Thanks so much for your email. The star rating is tied to the Field Partner, and not the loan. So when the star rating changes, it will change for all the loans regardless of status.
I'll be sure to pass along your feedback regarding changes in the star ratings. Please note that you can see an FAQ about MIFEX on their Field Partner page here -http://www.kiva.org/about/aboutPartner?id=7.
(salutation, end transmission. I replied:)
Thank you for your prompt and clear response. Yes, please pass along this:
1) The defaults-per-star statistics must reflect the conditions under which people made loans. Otherwise, they give actively inaccurate information to people who mean to invest.
If I had not gotten contradictory information and written in a question, I would have believed, based on everything on your site, that no loan from a 4 or 5 star org has ever defaulted in the history of Kiva. This is false advertising and doesn't give people the chance to do reasonable diligence research from your site. I know this will probably mean a big change to your underlying database (I'm a software tester), but it's absolutely essential.
Right now the information on http://www.kiva.org/about/risk/statistics/ is inaccurate enough that I'd rather you didn't show it in that format at all. When you get the new tracking system in place, if you cannot reconstruct the ratings history for existing defaults (particularly MIFEX, WEEC, and SEED), please offer the historical data separately with a disclaimer and start new data that only uses accurate ratings so the old stuff doesn't throw off the percentages.
2) Along with that, and I know I'm not the first person to ask for this, we need a ratings history for partners, which gives the dates that each rating or status was awarded and a one-sentence (at least) reason for the change. Example, "One year successful partnership" or "Completed fiscal audit" for an improved rating, "Recent increase in defaults" or "Investigating fiscal reporting practices" for a reduced rating. (It's not slander to say you're slowing things down while you look at the books.)
3) Please show statistics on the risk page defining how many defaults (and how much money defaulted) appear to have been due to the individual borrower's situation (political unrest, bad harvest, illness) versus partner organization situation (MIFEX, WEEC, SEED).
If I had known those statistics in advance instead of having to work them out by clever searches of your database, I would have made it a policy from day 1 to diversify as much as possible between partner organizations instead of spreading things over individual borrowers within the same partner. Frankly, knowing that individuals almost never default makes me feel even more strongly committed to investing and donating than I was before.
I know these statistics can be ugly and a bit embarrassing. I feel a lot more comfortable investing in a place that's perfectly transparent about the risks (even a 5-star sometimes turns out to be cooking the books) than a place that's trying to hide things even a little bit. Whether you mean to or not, right now your statistics pages are blatantly hiding things.
Thank you again for your prompt response to my question. Don't worry about responding to me again except to say that you've got it, and that my feedback will get to the right place. Thank you for the work you do.
(salutation, end transmission)
I still think they're doing a great thing, and I'm still very glad to be investing in their programs. The overall statistics are still the same (over time, if the past is like the future, you'll lose an average of 2.3% to defaults). I'm just making sure the details are all transparent. If you know anyone who's looking into Kiva, feel free to link to me or otherwise pass it along.
Belated Edit: The tracking is not as poor as first implied. I got a later response clarifying this, which hpet has pasted in a comment at the bottom. Check it out. They still have said in other places that their star ratings are not very precise, though.
The statistics at http://www.kiva.org/about/risk/statistics/ say that no loan from a 4 or 5 star organization has defaulted ever. As in ever. I thought that was pretty amazing. It would be, except it's not true, it's poor data tracking on their part.
I did a whole pile of digging around in their default statistics (I'm not worried about delayed payments, just payments that don't come through in the final tally). My findings are laid out in the comments here: http://gement.livejournal.com/96419.html and most of those findings are valid. However, based on the input from a pissed-off investor in MIFEX loans, also in those comments, I recognized that Kiva might not be doing a good job of data tracking and analysis.
(So I sent them this question...)
Subject: Our Field Partners - When ratings change, does it get tracked in the statistics?
I'm trying to understand how changed ratings get tracked in the site statistics on defaults, etc. I'm tracking the MIFEX case, of course, and I see that between May and November you were offering loans through them with a 1-star rating.
When everything defaulted, did the loans made when they were rated 3 stars stay in the 3 star statistics and the loans made when they were rated 1 star go in the 1 star statistics?
I'm asking because someone told me that he invested in MIFEX when they were rated 4 stars and lost money. Since you do not have ratings history (please add this as soon as possible), I cannot verify whether they were ever rated 4 stars. If a loan was made when they were 4 stars, and it defaulted, would it have ended up in the 4 star statistics, or moved down with the partner's adjusted rating?
(Obviously, I think the loan should be counted in the statistics for the star rating it was offered under. I just want to find out if that's how it works _now_, and if not, how soon you plan to fix that.)
(salutation, end transmission. Their reply, within one day...)
Thanks so much for your email. The star rating is tied to the Field Partner, and not the loan. So when the star rating changes, it will change for all the loans regardless of status.
I'll be sure to pass along your feedback regarding changes in the star ratings. Please note that you can see an FAQ about MIFEX on their Field Partner page here -http://www.kiva.org/about/aboutPartner?id=7.
(salutation, end transmission. I replied:)
Thank you for your prompt and clear response. Yes, please pass along this:
1) The defaults-per-star statistics must reflect the conditions under which people made loans. Otherwise, they give actively inaccurate information to people who mean to invest.
If I had not gotten contradictory information and written in a question, I would have believed, based on everything on your site, that no loan from a 4 or 5 star org has ever defaulted in the history of Kiva. This is false advertising and doesn't give people the chance to do reasonable diligence research from your site. I know this will probably mean a big change to your underlying database (I'm a software tester), but it's absolutely essential.
Right now the information on http://www.kiva.org/about/risk/statistics/ is inaccurate enough that I'd rather you didn't show it in that format at all. When you get the new tracking system in place, if you cannot reconstruct the ratings history for existing defaults (particularly MIFEX, WEEC, and SEED), please offer the historical data separately with a disclaimer and start new data that only uses accurate ratings so the old stuff doesn't throw off the percentages.
2) Along with that, and I know I'm not the first person to ask for this, we need a ratings history for partners, which gives the dates that each rating or status was awarded and a one-sentence (at least) reason for the change. Example, "One year successful partnership" or "Completed fiscal audit" for an improved rating, "Recent increase in defaults" or "Investigating fiscal reporting practices" for a reduced rating. (It's not slander to say you're slowing things down while you look at the books.)
3) Please show statistics on the risk page defining how many defaults (and how much money defaulted) appear to have been due to the individual borrower's situation (political unrest, bad harvest, illness) versus partner organization situation (MIFEX, WEEC, SEED).
If I had known those statistics in advance instead of having to work them out by clever searches of your database, I would have made it a policy from day 1 to diversify as much as possible between partner organizations instead of spreading things over individual borrowers within the same partner. Frankly, knowing that individuals almost never default makes me feel even more strongly committed to investing and donating than I was before.
I know these statistics can be ugly and a bit embarrassing. I feel a lot more comfortable investing in a place that's perfectly transparent about the risks (even a 5-star sometimes turns out to be cooking the books) than a place that's trying to hide things even a little bit. Whether you mean to or not, right now your statistics pages are blatantly hiding things.
Thank you again for your prompt response to my question. Don't worry about responding to me again except to say that you've got it, and that my feedback will get to the right place. Thank you for the work you do.
(salutation, end transmission)
I still think they're doing a great thing, and I'm still very glad to be investing in their programs. The overall statistics are still the same (over time, if the past is like the future, you'll lose an average of 2.3% to defaults). I'm just making sure the details are all transparent. If you know anyone who's looking into Kiva, feel free to link to me or otherwise pass it along.
