Reference: Communications of the ACM 43 (8) (August 2000) 24-25.

Does Typography Affect Proposal Assessment?

Daniel Berleant

Nesev Associates

Introduction

Experience from assisting in review of thirty proposals to a major federal funding agency suggests: mundane aspects of proposal formatting are correlated with proposal assessments. Correlation is not causality, but is suggestive. Our analysis also suggests some specific formatting guidelines.

Despite the small size of the sample of proposals analyzed, the analysis does suggest pragmatic formatting heuristics. The results also point up the need for a more comprehensive survey that would produce formatting guidance supported by statistically significant data on which proposals are funded and which are not.

Why do these apparent connections between mundane formatting and actual funding occur? Here are a few possibilities, any of which might apply to a given formatting heuristic.

1. Causality. A formatting decision may cause a proposal to be viewed more or less favorably, for example by making a more or less favorable initial impression on reviewers and program directors.
2. Aesthetics. An aesthetically pleasing proposal won’t hurt and probably would help.
3. Social conformity.  Unstated formatting norms might signal membership (or lack thereof) in the group consisting of the most fundable group of researchers.
4. Sophistication of the proposer. People with the experience and ability to write successful proposals may tend to make certain formatting choices differently from others, so that for example use of a relatively small font size might occur when the proposer simply has more to say. On this view, a formatting property might correlate with high proposal ratings, yet have nothing to do with them.

Publicizing the effects of formatting on proposal acceptance levels the playing field by helping researchers to make better formatting decisions, causing funding decisions to be based more on proposal content and less on proposal form. This is especially beneficial to inexperienced proposers, those who mistakenly favor the “wrong” formats, those who spend time and energy deciding format issues, and all who want to avoid the potential disadvantage of unfavorable proposal formats.

Without further ado, here are the analyses of the thirty proposals and the formatting heuristics the analyses suggest. A more comprehensive study is required to verify or revise what these data suggest.

Data and Analysis

Given that the body of the proposal narrative is in a serif font, are sans-serif section headings desirable? Table 1 shows that proposals rated as Not Competitive are relatively less likely to have sans-serif section headings. The table therefore suggests writing proposals with section headings in a sans-serif font.

Sans-serif headings?    Yes      No

Highly Competitive      2          5         

Competitive                  3          6         

Not Competitive          1          9         

Table 1. Considering the proposals written in a serif font, what is the effect of sans-serif section headings? (Only proposals being considered for primary funding under the program for which they were reviewed are tabulated.)

Table 2 shows that proposals rated as Not Competitive are the only ones to make the apparent misstep of numbering subsections of the narrative as sub-subsections of the overall proposal. The table therefore suggests numbering subsections of the narrative independently of the section structure of the proposal as a whole.

Subsection numbering Yes      No

Highly Competitive      0          6         

Competitive                  0          8         

Not Competitive          2          6         

Table 2. Although the proposal narrative is one of several distinct sections of a full proposal, some proposers numbered subsections of the narrative independently of the section structure of the rest of the proposal (i.e. they used sections 1, 2, etc.) while two numbered subsections of the narrative as sub-subsections of the proposal (i.e. if the narrative was section 8 of the proposal, sections of the narrative were numbered 8.1, 8.2, etc.) (Only proposals with numbered subsections in the narrative, and being considered for primary funding under the program are tabulated.)

Table 3 shows that proposal narratives written in 12 pt. text were disproportionately low in competitiveness, while the more competitive proposals tended to use smaller font sizes. (Exception: one non-competitive proposal used 9 pt. text.) The table therefore suggests writing proposals with narratives in 10 pt. text, and avoiding 12 pt. text.

Font size                   9          10        11        12

Highly Competitive  0   4          3          0         

Competitive              0         3          5          3         

Not Competitive       1         3          4          4         

Table 3. The text of the body of a proposal narrative is normally from 10 to 12 points in size. Although a type setting heuristic is to avoid over 80 characters per line, proposal narratives had a 1-column layout and even lines in 12 point proposals typically had over 80 characters. Despite this, 10 and 11 point text seemed to be associated with more competitive proposals.

Table 4 shows that the more competitive proposals tended to have a conclusion (or similarly named) section of the narrative. This is despite the lack of mention of a conclusion in the official guidelines for content of a proposal narrative. The table therefore suggests including a conclusion section in proposal narratives.

Conclusion                   Yes      No

Highly Competitive      3          4         

Competitive                  3          6         

Not Competitive           2          8         

Table 4. The effect of including a conclusion section in a proposal narrative. (Only proposals being considered for primary funding under the program are tabulated.)

Table 5 shows that every proposal rated as highly competitive had an abstract or other short introduction in the narrative. Because the overall proposal format required a separate section providing a summary of the proposal, some proposers might assume such an introductory section in the narrative is unnecessary. Proposers of the most competitive proposals, however, did not make that assumption. The table therefore suggests writing proposals with brief introductions (despite the lack of mention of this in the official guidelines for narrative content, a fact which might mislead some proposers to assume it is unnecessary or even to be avoided).

Abstract in Narrative?             Yes      No

Highly Competitive                  7          0         

Competitive                              9          2         

Not Competitive                      11         1         

Table 5. The effect of including a brief introductory section.

Proposals included a summary section separate from the narrative section. A proposer might then reuse this section as part of the narrative. However, Table 6 shows that proposals rated as Highly Competitive had the greatest tendency to avoid duplication between the proposal summary section and the proposal narrative section.

Summary reused?        Yes      Partly    No

Highly Competitive      0          4           3       

Competitive                  0         10          1

Not Competitive          1          7           4

Table 6. Is the summary section of the proposal reused verbatim in the proposal narrative? (Yes: nearly so; Partly: part is reused or sizeable pieces are reused in different places in the narrative; No: little or none is reprinted.)

Table 7 suggests that the more competitive a proposal is, the more likely it is to be submitted early. This could reflect a lack of deleterious "last minute rush" compromises on quality.

# Days early                  >=3      2          1          0

Highly Competitive           2    0          1          4

Competitive                       0      2          1          8

Not Competitive                1      0          3          8

Table 7. Does submitting proposals early have an effect? (All proposals were submitted electronically, which aided in tabulating this information.)

Table 8 suggests that following the suggested outline had no clear benefit. Indeed, the data suggest that following the outline very closely (and therefore having no conclusion, for example) is deleterious.

Standard outline?

Yes       Mostly Some   A little No

Highly Competitive

0            2          2          1          2

Competitive

2            4          2          0          3

Not Competitive

2            1          3          3          3

Table 8. The proposal call suggested an outline, which was often not followed to a greater or lesser degree. Yes signifies the suggested outline was followed closely, No signifies it wasn’t followed at all, etc.

Table 9 suggests that use of the term “we” is preferable to use of “I” or the third person. Over half the proposals ranked as Highly Competitive used “we,” compared to less than  half of the proposals in the “I” and 3^rd person categories.

I/We/3rd                       I           We       3rd

Highly Competitive      1          4          2         

Competitive                  5          2          2

Not Competitive          3          4          3

Table 9. Tabulation of how proposers referred to themselves. “3^rd” refers to the third person, such as when the proposal used phrases like “the proposer” or the proposer’s name. (Only proposals being considered for primary funding under the program are tabulated.)

Conclusion

The data and analysis here suggest formatting guidelines to use when writing proposals. These results are more applicable to proposals in the computing field than other areas, because the proposals analysed were in computing proposals. As much or even more, the results indicate a need for more rigorous analyses using more data that will result in clear formatting guidelines. While the most successful proposers are probably already using the “right” formatting guidelines, many others do not. If they did, this would help level the playing field and free proposal review from the distractions and noise introduced by variations in form, independent of content –  and content is what proposal review should be all about.

Daniel Berleant (berleant@iastate.edu) is an Associate Prof. In the Electrical and Computer Engineering Dept., Iowa State University.

Addendum (2/2/03)

Figure captions, note 1. Six had the term “Figure” followed by a period, of which zero (0) were rated non-competitive. Fifteen had the term “Figure” followed by a colon, of which nine (9) were rated non-competitive. Suggestive isn’t it?

Figure captions, note 2. Although virtually all captions began with a sentence fragment, some ended with a period, while others did not end it with a period. Five non-competitive proposals ended captions with periods, and five non-competitive proposals did not. However, nine (9) competitive or highly competitive proposals ended captions in periods while three (3) did not. Good grammar did not seem to win, here. Also suggestive.

Figure captions, note 3. Some proposals did not have figures or had figures without captions. One used a dash instead of a colon or period. Some had captions that were quite convoluted, punctuation-wise. I’ve had reviewers complain about long captions, demanding that the explanations appear in the text. Yet, from a modularity standpoint, one might expect that it would be better for figures to be self-contained, implying longer captions. Innumerable analyses could be done. One NSF program director, when approached with the idea that a continuation of the research described in this paper might be funded, to help level the playing field, had an interesting reaction. He had actually read this CACM article, and was quite angry about it, and even angry about the entire issue that this article appeared in, claiming it was a weird issue (or something to that effect). As for funding this kind of work, he felt that there would be no money to analyze the “chicken entrails” (his words) of NSF.

Go figure.

Naturally, funding agency agents will claim and want to believe that only content counts in funding decisions, but that’s like saying what you wear doesn’t matter. Just try giving a formal presentation in a pink T-shirt (esp. if you are male). I’ve seen it happen, and the speaker was severely chastised. Not for the T-shirt, but draw your own conclusions.

Contact the author with observations or questions.

Addendum (9/29/03)

Does the first sentence of the narrative try to grab the reader's attention?

Beginning tries to grab?	Non-competitive	Competitive	Highly competitive
Yes	3	8	5
No	8	3	4

The following questions refer to the first n paragraphs of the narrative for which the end of the nth paragraph is before the first half page of text. The half page measurement does not count space taken up by title type material, and does not correct for type size or margins.

Is a gap in current knowledge described?

Gap?	Non-competitive	Competitive	Highly competitive
Yes	6	10	7
No	5	1	2

Is the proposed work described?

Says what is proposed?	Non-competitive	Competitive	Highly competitive
Yes	4	5	7
No	7	6	2

Is an overview of the present state of knowledge described?

OverviewNow?	Non-competitive	Competitive	Highly competitive
Yes	10	11	5
No	1	0	4

Is a historical overview of the area described?

Historical overview?	Non-competitive	Competitive	Highly competitive
yes	4	3	3
no	7	8	6

Is either (or both) historical overview or the present state of knowledge described?

Highly competitive: Yes=5, No=4. Not one proposal that failed to give an overview of the present state of knowledge gave a historical overview.